Chapter 9. System tips

The simple use of script(1) (see Section 1.4.9, “Recording the shell activities”) to record shell activity produces a file with control characters. This can be avoided by using col(1) as the following.

$ script
Script started, file is typescript

Do whatever … and press Ctrl-D to exit script.

$ col -bx < typescript > cleanedfile
$ vim cleanedfile

There are alternative methods to record the shell activities:

In Section 1.4.2, “Customizing bash”, 2 tips to allow quick navigation around directories are described: $CDPATH and mc.

If you use fuzzy text filter program, you can do without typing the exact path. For fzf, include following in ~/.bashrc.

FZF_KEYBINDINGS_PATH=/usr/share/doc/fzf/examples/key-bindings.bash
if [ -f $FZF_KEYBINDINGS_PATH ]; then
  . $FZF_KEYBINDINGS_PATH
fi

For example:

These Ex-mode commands can be included in user's vimrc file, traditional "~/.vimrc" or git-friendly "~/.vim/vimrc". Here is a very simple example ^[2]:

""" Generic baseline Vim and Neovim configuration (~/.vimrc)
"""   - For NeoVim, use "nvim -u ~/.vimrc [filename]"
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
let mapleader = ' '             " :h mapleader
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
set nocompatible                " :h 'cp -- sensible (n)vim mode
syntax on                       " :h :syn-on
filetype plugin indent on       " :h :filetype-overview
set encoding=utf-8              " :h 'enc (default: latin1) -- sensible encoding
""" current vim option value can be verified by :set encoding?
set backspace=indent,eol,start  " :h 'bs (default: nobs) -- sensible BS
set statusline=%<%f%m%r%h%w%=%y[U+%04B]%2l/%2L=%P,%2c%V
set listchars=eol:¶,tab:⇄\ ,extends:↦,precedes:↤,nbsp:␣
set viminfo=!,'100,<5000,s100,h " :h 'vi -- bigger copy buffer etc.
""" Pick "colorscheme" from blue darkblue default delek desert elflord evening
""" habamax industry koehler lunaperche morning murphy pablo peachpuff quiet ron
""" shine slate torte zellner
colorscheme industry
"colorscheme default
set scrolloff=5                 " :h 'scr -- show 5 lines around cursor
set laststatus=2                " :h 'ls (default 1)  k
""" boolean options can be unset by prefixing "no"
set ignorecase                  " :h 'ic
set smartcase                   " :h 'scs
set autoindent                  " :h 'ai
set smartindent                 " :h 'si
set nowrap                      " :h 'wrap
"set list                        " :h 'list (default nolist)
set noerrorbells                " :h 'eb
set novisualbell                " :h 'vb
set t_vb=                       " :h 't_vb -- termcap visual bell
set spell                       " :h 'spell
set spelllang=en_us,cjk         " :h 'spl -- english spell, ignore CJK
set clipboard=unnamedplus       " :h 'cb -- cut/copy/paste with other app
set hidden                      " :h 'hid
set autowrite                   " :h 'aw

The keymap of vim can be changed in user's vimrc file. E.g.:

	Caution
	Don't try to change the default key bindings without very good reasons.

""" Popular mappings (imitating LazyVim etc.)
""" Window moves without using CTRL-W which is dangerous in INSERT mode
nnoremap <C-H> <C-W>h
nnoremap <C-J> <C-W>j
nnoremap <C-K> <C-W>k
silent! nnoremap <C-L> <C-W>l
""" Window resize
nnoremap <C-LEFT> <CMD>vertical resize -2<CR>
nnoremap <C-DOWN> <CMD>resize -2<CR>
nnoremap <C-UP> <CMD>resize +2<CR>
nnoremap <C-RIGHT> <CMD>vertical resize +2<CR>
""" Clear hlsearch with <ESC> (<C-L> is mapped as above)
nnoremap <ESC> <CMD>noh<CR><ESC>
inoremap <ESC> <CMD>noh<CR><ESC>
""" center after jump next
nnoremap n nzz
nnoremap N Nzz
""" fast "jk" to get out of INSERT mode (<ESC>)
inoremap  jk <CMD>noh<CR><ESC>
""" fast "<ESC><ESC>" to get out of TERM mode (CTRL-\ CTRL-N)
tnoremap <ESC><ESC> <C-\><C-N>
""" fast "jk" to get out of TERM mode (CTRL-\ CTRL-N)
tnoremap jk <C-\><C-N>
""" previous/next trouble/quickfix item
nnoremap [q <CMD>cprevious<CR>
nnoremap ]q <CMD>cnext<CR>
""" buffers
nnoremap <S-H> <CMD>bprevious<CR>
nnoremap <S-L> <CMD>bnext<CR>
nnoremap [b <CMD>bprevious<CR>
nnoremap ]b <CMD>bnext<CR>
""" Add undo break-points
inoremap  , ,<C-G>u
inoremap  . .<C-G>u
inoremap  ; ;<C-G>u
""" save file
inoremap <C-S> <CMD>w<CR><ESC>
xnoremap <C-S> <CMD>w<CR><ESC>
nnoremap <C-S> <CMD>w<CR><ESC>
snoremap <C-S> <CMD>w<CR><ESC>
""" better indenting
vnoremap < <gv
vnoremap > >gv
""" terminal (Somehow under Linux, <C-/> becomes <C-_> in Vim)
nnoremap <C-_> <CMD>terminal<CR>
"nnoremap <C-/> <CMD>terminal<CR>
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
if ! has('nvim')
""" Toggle paste mode with <SPACE>p for Vim (no need for Nvim)
set pastetoggle=<leader>p
""" nvim default mappings for Vim.  See :h default-mappings in nvim
""" copy to EOL (no delete) like D for d
noremap Y y$
""" sets a new undo point before deleting
inoremap <C-U> <C-G>u<C-U>
inoremap <C-W> <C-G>u<C-W>
""" <C-L> is re-purposed as above
""" execute the previous macro recorded with Q
nnoremap Q @@
""" repeat last substitute and *KEEP* flags
nnoremap & :&&<CR>
""" search visual selected string for visual mode
xnoremap * y/\V<C-R>"<CR>
xnoremap # y?\V<C-R>"<CR>
endif

In order for the above keybindings to function properly, the terminal program needs to be configured to generate "ASCII DEL" for Backspace-key and "Escape sequence" for Delete-key.

Other miscellaneous configuration can be changed in user's vimrc file. E.g.:

""" Use faster 'rg' (ripgrep package) for :grep
if executable("rg")
  set grepprg=rg\ --vimgrep\ --smart-case
  set grepformat=%f:%l:%c:%m
endif
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
""" Retain last cursor position :h '"
augroup RetainLastCursorPosition
  autocmd!
  autocmd BufReadPost *
    \ if line("'\"") > 0 && line ("'\"") <= line("$") |
    \   exe "normal! g'\"" |
    \ endif
augroup END
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
""" Force to use underline for spell check results
augroup SpellUnderline
  autocmd!
  autocmd ColorScheme * highlight SpellBad term=Underline gui=Undercurl
  autocmd ColorScheme * highlight SpellCap term=Underline gui=Undercurl
  autocmd ColorScheme * highlight SpellLocal term=Underline gui=Undercurl
  autocmd ColorScheme * highlight SpellRare term=Underline gui=Undercurl
augroup END
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
""" highlight tailing spaces except when typing as red (set after colorscheme)
highlight TailingWhitespaces ctermbg=red guibg=red
""" \s\+     1 or more whitespace character: <Space> and <Tab>
""" \%#\@<!  Matches with zero width if the cursor position does NOT match.
match TailingWhitespaces /\s\+\%#\@<!$/

Plugin packages in the vim-scripts package can be enabled using user's vimrc file. E.g.:

packadd! secure-modelines
packadd! winmanager
" IDE-like UI for files and buffers with <space>w
nnoremap <leader>w         :WMToggle<CR>

The new native Vim package system works nicely with "git" and "git submodule". One such example configuration can be found at my git repository: dot-vim. This does essentially:

For more, please start vim with "vim --startuptime vimstart.log" to check actual execution sequence and time spent for each step.

It is quite confusing to see too many ways^[3] to manage and load these external packages to vim. Checking the original information is the best cure.

See Section 3.4, “The system message” and Section 3.3, “The kernel message”.

Although pager tools such as more(1) and less(1) (see Section 1.4.5, “The pager”) and custom tools for highlighting and formatting (see Section 11.1.8, “Highlighting and formatting plain text data”) can display text data nicely, general purpose editors (see Section 1.4.6, “The text editor”) are most versatile and customizable.

	Tip
	For vim(1) and its pager mode alias view(1), ":set hls" enables highlighted search.

The default display format of time and date by the "ls -l" command depends on the locale (see Section 1.2.6, “Timestamps” for value). The "$LANG" variable is referred first and it can be overridden by the "$LC_TIME" or "$LC_ALL" exported environment variables.

The actual default display format for each locale depends on the version of the standard C library (the libc6 package) used. I.e., different releases of Debian had different defaults. For iso-formats, see ISO 8601.

If you really wish to customize this display format of time and date beyond the locale, you should set the time style value by the "--time-style" argument or by the "$TIME_STYLE" value (see ls(1), date(1), "info coreutils 'ls invocation'").

	Tip
	You can eliminate typing long option on commandline using command alias (see Section 1.5.9, “Command alias”): alias ls='ls --time-style=+%d.%m.%y %H:%M'

Shell echo to most modern terminals can be colorized using ANSI escape code (see "/usr/share/doc/xterm/ctlseqs.txt.gz").

For example, try the following

$ RED=$(printf "\x1b[31m")
$ NORMAL=$(printf "\x1b[0m")
$ REVERSE=$(printf "\x1b[7m")
$ echo "${RED}RED-TEXT${NORMAL} ${REVERSE}REVERSE-TEXT${NORMAL}"

For Vim, as follows.

For Emacs, as follows.

There are specialized tools to record changes in configuration files with help of DVCS and to make system snapshots on Btrfs.

You may also think about local script Section 10.2.3, “Backup tips” approach.

$ sudo fuser -v /var/log/mail.log
                     USER        PID ACCESS COMMAND
/var/log/mail.log:   root       2946 F.... rsyslogd

$ sudo fuser -v smtp/tcp
                     USER        PID ACCESS COMMAND
smtp/tcp:            Debian-exim   3379 F.... exim4

Now you know your system runs exim4(8) to handle TCP connections to SMTP port (25).

for x in *.ext; do if [ -f "$x"]; then command "$x" ; fi; done

find . -type f -maxdepth 1 -name '*.ext' -print0 | xargs -0 -n 1 command

find . -type f -maxdepth 1 -name '*.ext' -exec command '{}' \;

find . -type f -maxdepth 1 -name '*.ext' -exec sh -c "command '{}' && echo 'successful'" \;

The above examples are written to ensure proper handling of funny file names such as ones containing spaces. See Section 10.1.5, “Idioms for the selection of files” for more advance uses of find(1).

For the command-line interface (CLI), the first program with the matching name found in the directories specified in the $PATH environment variable is executed. See Section 1.5.3, “The "$PATH" variable”.

For the graphical user interface (GUI) compliant to the freedesktop.org standards, the *.desktop files in the /usr/share/applications/ directory provide necessary attributes for the GUI menu display of each program. Each package which is compliant to Freedesktop.org's xdg menu system installs its menu data provided by "*.desktop" under "/usr/share/applications/". Modern desktop environments which are compliant to Freedesktop.org standard use these data to generate their menu using the xdg-utils package. See "/usr/share/doc/xdg-utils/README".

For example, the chromium.desktop file defines attributes for the "Chromium Web Browser" such as "Name" for the program name, "Exec" for the program execution path and arguments, "Icon" for the icon used, etc. (see the Desktop Entry Specification) as follows:

[Desktop Entry]
Version=1.0
Name=Chromium Web Browser
GenericName=Web Browser
Comment=Access the Internet
Comment[fr]=Explorer le Web
Exec=/usr/bin/chromium %U
Terminal=false
X-MultipleArgs=false
Type=Application
Icon=chromium
Categories=Network;WebBrowser;
MimeType=text/html;text/xml;application/xhtml_xml;x-scheme-handler/http;x-scheme-handler/https;
StartupWMClass=Chromium
StartupNotify=true

This is an oversimplified description. The *.desktop files are scanned as follows.

The desktop environment sets $XDG_DATA_HOME and $XDG_DATA_DIR environment variables. For example, under the GNOME 3:

So the base directories (see XDG Base Directory Specification) and the applications directories are as follows.

The *.desktop files are scanned in these applications directories in this order.

	Tip
	A user custom GUI menu entry can be created by adding a *.desktop file in the $HOME/.local/share/applications/ directory.

	Tip
	The "Exec=..." line isn't parsed by the shell. Use the env(1) command if environment variables need to be set.

	Tip
	Similarly, if a *.desktop file is created in the autostart directory under these base directories, the specified program in the *.desktop file is executed automatically when the desktop environment is started. See Desktop Application Autostart Specification.

	Tip
	Similarly, if a *.desktop file is created in the $HOME/Desktop directory and the Desktop environment is configured to support the desktop icon launcher feature, the specified program in it is executed upon clicking the icon. Please note that the actual name of the $HOME/Desktop directory is locale dependent. See xdg-user-dirs-update(1).

Use cron(8) to schedule tasks regularly. See crontab(1) and crontab(5).

You can schedule to run processes as a normal user, e.g. foo by creating a crontab(5) file as "/var/spool/cron/crontabs/foo" with "crontab -e" command.

Here is an example of a crontab(5) file.

# use /usr/bin/sh to run commands, no matter what /etc/passwd says
SHELL=/bin/sh
# mail any output to paul, no matter whose crontab this is
MAILTO=paul
# Min Hour DayOfMonth Month DayOfWeek command (Day... are OR'ed)
# run at 00:05, every day
5  0  *  * *   $HOME/bin/daily.job >> $HOME/tmp/out 2>&1
# run at 14:15 on the first of every month -- output mailed to paul
15 14 1  * *   $HOME/bin/monthly
# run at 22:00 on weekdays(1-5), annoy Joe. % for newline, last % for cc:
0 22 *   * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?%.%%
23 */2 1 2 *   echo "run 23 minutes after 0am, 2am, 4am ..., on Feb 1"
5  4 *   * sun echo "run at 04:05 every Sunday"
# run at 03:40 on the first Monday of each month
40 3 1-7 * *   [ "$(date +%a)" == "Mon" ] && command -args

	Tip
	For the system not running continuously, install the anacron package to schedule periodic commands at the specified intervals as closely as machine-uptime permits. See anacron(8) and anacrontab(5).

	Tip
	For scheduled system maintenance scripts, you can run them periodically from root account by placing such scripts in "/etc/cron.hourly/", "/etc/cron.daily/", "/etc/cron.weekly/", or "/etc/cron.monthly/". Execution timings of these scripts can be customized by "/etc/crontab" and "/etc/anacrontab".

Systemd has low level capability to schedule programs to run without cron daemon. For example, /lib/systemd/system/apt-daily.timer and /lib/systemd/system/apt-daily.service set up daily apt download activities. See systemd.timer(5) .

Systemd can schedule program not only on the timer event but also on the mount event. See Section 10.2.3.3, “Timer event triggered backup” and Section 10.2.3.2, “Mount event triggered backup” for examples.

See more on Linux kernel user’s and administrator’s guide » Linux Magic System Request Key Hacks

	Tip
	From SSH terminal etc., you can use the Alt-SysRq feature by writing to the "/proc/sysrq-trigger". For example, "echo s > /proc/sysrq-trigger; echo u > /proc/sysrq-trigger" from the root shell prompt syncs and umounts all mounted filesystems.

The current (2021) Debian amd64 Linux kernel has /proc/sys/kernel/sysrq=438=0b110110110:

For the PCI-like devices (AGP, PCI-Express, CardBus, ExpressCard, etc.), lspci(8) (probably with "-nn" option) is a good start for the hardware identification.

Alternatively, you can identify the hardware by reading contents of "/proc/bus/pci/devices" or browsing directory tree under "/sys/bus/pci" (see Section 1.2.12, “procfs and sysfs”).

Here, ACPI is a newer framework for the power management system than APM.

	Tip
	CPU frequency scaling on modern system is governed by kernel modules such as acpi_cpufreq.

# date MMDDhhmmCCYY
# hwclock --utc --systohc
# hwclock --show

Times are normally displayed in the local time on the Debian system but the hardware and system time usually use UTC(GMT).

If the hardware time is set to UTC, change the setting to "UTC=yes" in the "/etc/default/rcS".

The following reconfigure the timezone used by the Debian system.

# dpkg-reconfigure tzdata

If you wish to update system time via network, consider to use the NTP service with the packages such as ntp, ntpdate, and chrony.

	Tip
	Under systemd, use systemd-timesyncd for the network time synchronization instead. See systemd-timesyncd(8).

See the following.

	Tip
	ntptrace(8) in the ntp package can trace a chain of NTP servers back to the primary source.

Device drivers for sound cards for current Linux are provided by Advanced Linux Sound Architecture (ALSA). ALSA provides emulation mode for previous Open Sound System (OSS) for compatibility.

Application softwares may be configured not only to access sound devices directly but also to access them via some standardized sound server system. Currently, PulseAudio, JACK, and PipeWire are used as sound server system. See Debian wiki page on Sound for the latest situation.

There is usually a common sound engine for each popular desktop environment. Each sound engine used by the application can choose to connect to different sound servers.

	Tip
	Use "cat /dev/urandom > /dev/audio" or speaker-test(1) to test speaker (^C to stop).

	Tip
	If you can not get sound, your speaker may be connected to a muted output. Modern sound system has many outputs. alsamixer(1) in the alsa-utils package is useful to configure volume and mute settings.

For disk partition configuration, although fdisk(8) has been considered standard, parted(8) deserves some attention. "Disk partitioning data", "partition table", "partition map", and "disk label" are all synonyms.

Older PCs use the classic Master Boot Record (MBR) scheme to hold disk partitioning data in the first sector, i.e., LBA sector 0 (512 bytes).

Recent PCs with Unified Extensible Firmware Interface (UEFI), including Intel-based Macs, use GUID Partition Table (GPT) scheme to hold disk partitioning data not in the first sector.

Although fdisk(8) has been standard for the disk partitioning tool, parted(8) is replacing it.

	Caution
	Although parted(8) claims to create and to resize filesystem too, it is safer to do such things using best maintained specialized tools such as mkfs(8) (mkfs.msdos(8), mkfs.ext2(8), mkfs.ext3(8), mkfs.ext4(8), …) and resize2fs(8).

	Note
	In order to switch between GPT and MBR, you need to erase first few blocks of disk contents directly (see Section 9.8.6, “Clearing file contents”) and use "parted /dev/sdx mklabel gpt" or "parted /dev/sdx mklabel msdos" to set it. Please note "msdos" is use here for MBR.

LVM2 is a logical volume manager for the Linux kernel. With LVM2, disk partitions can be created on logical volumes instead of the physical harddisks.

LVM requires the following.

Please start learning LVM2 from the following manpages.

For ext4 filesystem, the e2fsprogs package provides the following.

The mkfs(8) and fsck(8) commands are provided by the e2fsprogs package as front-ends to various filesystem dependent programs (mkfs.fstype and fsck.fstype). For ext4 filesystem, they are mkfs.ext4(8) and fsck.ext4(8) (they are symlinked to mke2fs(8) and e2fsck(8)).

Similar commands are available for each filesystem supported by Linux.

Tip

Ext4 filesystem is the default filesystem for the Linux system and strongly recommended to use it unless you have some specific reasons not to. Btrfs status can be found at Debian wiki on btrfs and kernel.org wiki on btrfs. It is expected to be the next default filesystem after the ext4 filesystem. Some tools allow access to filesystem without Linux kernel support (see Section 9.8.2, “Manipulating files without mounting disk”).

Solid state drive (SSD) is auto detected now.

Reduce unnecessary disk accesses to prevent disk wear out by mounting "tmpfs" on volatile data path in /etc/fstab.

You can monitor and log your hard disk which is compliant to SMART with the smartd(8) daemon.

	Tip
	The smartd(8) daemon can be customized with the /etc/smartd.conf file including how to be notified of warnings.

For partitions created on Logical Volume Manager (LVM) (Linux feature) at install time, they can be resized easily by concatenating extents onto them or truncating extents from them over multiple storage devices without major system reconfiguration.

If you have usable space in another partition (e.g., "/path/to/empty" and "/path/to/work"), you can create a directory in it and stack that on to an old directory (e.g., "/path/to/old") where you need space using the OverlayFS for Linux kernel 3.18 or newer (Debian Stretch 9.0 or newer).

$ sudo mount -t overlay overlay \
  -olowerdir=/path/to/old-dir,upperdir=/path/to/empty,workdir=/path/to/work

Here, "/path/to/empty" and "/path/to/work" should be on the RW-enabled partition to write on "/path/to/old".

# cp /dev/sdb disk.img
# dd if=/dev/sdb of=disk.img

The disk image of the traditional PC's master boot record (MBR) (see Section 9.6.2, “Disk partition configuration”) which reside on the first sector on the primary IDE disk can be made by using dd(1) by the following.

# dd if=/dev/hda of=mbr.img bs=512 count=1
# dd if=/dev/hda of=mbr-nopart.img bs=446 count=1
# dd if=/dev/hda of=mbr-part.img skip=446 bs=1 count=66

If you have an SCSI or serial ATA device as the boot disk, substitute "/dev/hda" with "/dev/sda".

If you are making an image of a disk partition of the original disk, substitute "/dev/hda" with "/dev/hda1" etc.

The disk image "partition.img" containing a single partition image can be mounted and unmounted by using the loop device as follows.

# losetup --show -f partition.img
/dev/loop0
# mkdir -p /mnt/loop0
# mount -t auto /dev/loop0 /mnt/loop0
...hack...hack...hack
# umount /dev/loop0
# losetup -d /dev/loop0

This can be simplified as follows.

# mkdir -p /mnt/loop0
# mount -t auto -o loop partition.img /mnt/loop0
...hack...hack...hack
# umount partition.img

Each partition of the disk image "disk.img" containing multiple partitions can be mounted by using the loop device.

# losetup --show -f -P disk.img
/dev/loop0
# ls -l /dev/loop0*
brw-rw---- 1 root disk   7,  0 Apr  2 22:51 /dev/loop0
brw-rw---- 1 root disk 259, 12 Apr  2 22:51 /dev/loop0p1
brw-rw---- 1 root disk 259, 13 Apr  2 22:51 /dev/loop0p14
brw-rw---- 1 root disk 259, 14 Apr  2 22:51 /dev/loop0p15
# fdisk -l /dev/loop0
Disk /dev/loop0: 2 GiB, 2147483648 bytes, 4194304 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6A1D9E28-C48C-2144-91F7-968B3CBC9BD1

Device         Start     End Sectors  Size Type
/dev/loop0p1  262144 4192255 3930112  1.9G Linux root (x86-64)
/dev/loop0p14   2048    8191    6144    3M BIOS boot
/dev/loop0p15   8192  262143  253952  124M EFI System

Partition table entries are not in disk order.
# mkdir -p /mnt/loop0p1
# mkdir -p /mnt/loop0p15
# mount -t auto /dev/loop0p1 /mnt/loop0p1
# mount -t auto /dev/loop0p15 /mnt/loop0p15
# mount |grep loop
/dev/loop0p1 on /mnt/loop0p1 type ext4 (rw,relatime)
/dev/loop0p15 on /mnt/loop0p15 type vfat (rw,relatime,fmask=0002,dmask=0002,allow_utime=0020,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro)
...hack...hack...hack
# umount /dev/loop0p1
# umount /dev/loop0p15
# losetup -d /dev/loop0

Alternatively, similar effects can be done by using the device mapper devices created by kpartx(8) from the kpartx package as follows.

# kpartx -a -v disk.img
add map loop0p1 (253:0): 0 3930112 linear 7:0 262144
add map loop0p14 (253:1): 0 6144 linear 7:0 2048
add map loop0p15 (253:2): 0 253952 linear 7:0 8192
# fdisk -l /dev/loop0
Disk /dev/loop0: 2 GiB, 2147483648 bytes, 4194304 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6A1D9E28-C48C-2144-91F7-968B3CBC9BD1

Device         Start     End Sectors  Size Type
/dev/loop0p1  262144 4192255 3930112  1.9G Linux root (x86-64)
/dev/loop0p14   2048    8191    6144    3M BIOS boot
/dev/loop0p15   8192  262143  253952  124M EFI System

Partition table entries are not in disk order.
# ls -l /dev/mapper/
total 0
crw------- 1 root root 10, 236 Apr  2 22:45 control
lrwxrwxrwx 1 root root       7 Apr  2 23:19 loop0p1 -> ../dm-0
lrwxrwxrwx 1 root root       7 Apr  2 23:19 loop0p14 -> ../dm-1
lrwxrwxrwx 1 root root       7 Apr  2 23:19 loop0p15 -> ../dm-2
# mkdir -p /mnt/loop0p1
# mkdir -p /mnt/loop0p15
# mount -t auto /dev/mapper/loop0p1 /mnt/loop0p1
# mount -t auto /dev/mapper/loop0p15 /mnt/loop0p15
# mount |grep loop
/dev/loop0p1 on /mnt/loop0p1 type ext4 (rw,relatime)
/dev/loop0p15 on /mnt/loop0p15 type vfat (rw,relatime,fmask=0002,dmask=0002,allow_utime=0020,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro)
...hack...hack...hack
# umount /dev/mapper/loop0p1
# umount /dev/mapper/loop0p15
# kpartx -d disk.img

$ dd bs=1 count=0 if=/dev/zero of=disk.img seek=5G

You can create an ext4 filesystem on this disk image "disk.img" using the loop device as follows.

# losetup --show -f disk.img
/dev/loop0
# mkfs.ext4 /dev/loop0
...hack...hack...hack
# losetup -d /dev/loop0
$ du  --apparent-size -h disk.img
5.0G  disk.img
$ du -h disk.img
83M disk.img

For "disk.img", its file size is 5.0 GiB and its actual disk usage is mere 83MiB. This discrepancy is possible since ext4 can hold sparse file.

	Tip
	The actual disk usage of sparse file grows with data which are written to it.

Using similar operation on devices created by the loop device or the device mapper devices as Section 9.7.3, “Mounting the disk image file”, you can partition this disk image "disk.img" using parted(8) or fdisk(8), and can create filesystem on it using mkfs.ext4(8), mkswap(8), etc.

The ISO9660 image file, "cd.iso", from the source directory tree at "source_directory" can be made using genisoimage(1) provided by cdrkit by the following.

#  genisoimage -r -J -T -V volume_id -o cd.iso source_directory

Similarly, the bootable ISO9660 image file, "cdboot.iso", can be made from debian-installer like directory tree at "source_directory" by the following.

#  genisoimage -r -o cdboot.iso -V volume_id \
   -b isolinux/isolinux.bin -c isolinux/boot.cat \
   -no-emul-boot -boot-load-size 4 -boot-info-table source_directory

Here Isolinux boot loader (see Section 3.1.2, “Stage 2: the boot loader”) is used for booting.

You can calculate the md5sum value and make the ISO9660 image directly from the CD-ROM device as follows.

$ isoinfo -d -i /dev/cdrom
CD-ROM is in ISO 9660 format
...
Logical block size is: 2048
Volume size is: 23150592
...
# dd if=/dev/cdrom bs=2048 count=23150592 conv=notrunc,noerror | md5sum
# dd if=/dev/cdrom bs=2048 count=23150592 conv=notrunc,noerror > cd.iso

	Warning
	You must carefully avoid ISO9660 filesystem read ahead bug of Linux as above to get the right result.

Software RAID systems offered by the Linux kernel provide data redundancy in the kernel filesystem level to achieve high levels of storage reliability.

There are tools to add data redundancy to files in application program level to achieve high levels of storage reliability, too.

$ ls -li
total 0
2738405 -rw-r--r-- 1 root root 0 2008-09-15 20:21 bar
2738404 -rw-r--r-- 2 root root 0 2008-09-15 20:21 baz
2738404 -rw-r--r-- 2 root root 0 2008-09-15 20:21 foo

Both "baz" and "foo" have link counts of "2" (>1) showing them to have hardlinks. Their inode numbers are common "2738404". This means they are the same hardlinked file. If you do not happen to find all hardlinked files by chance, you can search it by the inode, e.g., "2738404" as the following.

# find /path/to/mount/point -xdev -inum 2738404

You can encrypt contents of removable mass devices, e.g. USB memory stick on "/dev/sdx", using dm-crypt/LUKS. You simply format it as the following.

# fdisk /dev/sdx
... "n" "p" "1" "return" "return" "w"
# cryptsetup luksFormat /dev/sdx1
...
# cryptsetup open /dev/sdx1 secret
...
# ls -l /dev/mapper/
total 0
crw-rw---- 1 root root  10, 60 2021-10-04 18:44 control
lrwxrwxrwx 1 root root       7 2021-10-04 23:55 secret -> ../dm-0
# mkfs.vfat /dev/mapper/secret
...
# cryptsetup close secret

Then, it can be mounted just like normal one on to "/media/username/disk_label", except for asking password (see Section 10.1.7, “Removable storage device”) under modern desktop environment using the udisks2 package. The difference is that every data written to it is encrypted. The password entry may be automated using keyring (see Section 10.3.6, “Password keyring”).

You may alternatively format media in different filesystem, e.g., ext4 with "mkfs.ext4 /dev/mapper/sdx1". If btrfs is used instead, the udisks2-btrfs package needs to be installed. For these filesystems, the file ownership and permissions may need to be configured.

See "The Linux kernel user’s and administrator’s guide » The kernel’s command-line parameters" for the detail.

If you use initrd in Section 3.1.2, “Stage 2: the boot loader”, make sure to read the related information in initramfs-tools(8), update-initramfs(8), mkinitramfs(8) and initramfs.conf(5).

	Warning
	Do not put symlinks to the directories in the source tree (e.g. "/usr/src/linux*") from "/usr/include/linux" and "/usr/include/asm" when compiling the Linux kernel source. (Some outdated documents suggest this.)

Note

When compiling the latest Linux kernel on the Debian stable system, the use of backported latest tools from the Debian unstable may be needed. module-assistant(8) (or its short form m-a) helps users to build and install module package(s) easily for one or more custom kernels. The dynamic kernel module support (DKMS) is a new distribution independent framework designed to allow individual kernel modules to be upgraded without changing the whole kernel. This is used for the maintenance of out-of-tree modules. This also makes it very easy to rebuild modules as you upgrade kernels.

The firmware is the code or data loaded on the device attach to the target system (e.g., CPU microcode, rendering code running on GPU, or FPGA / CPLD data, …). Some firmware packages are available as free software but many firmware packages are not available as free software since they contain sourceless binary data. Installing these firmware data is essential for the device to function as expected.

Please note that access to non-free-firmware packages are provided by the official installation media to offer functional installation experience to the user since Debian 12 Bookworm. The non-free-firmware area is described in Section 2.1.5, “Debian archive basics”.

Please also note that the firmware data downloaded by fwupd from Linux Vendor Firmware Service and loaded to the running Linux kernel may be non-free.

There are several virtualization and emulation tool platforms.

The container virtualization uses Section 4.7.5, “Linux security features” and is the backend technology of Section 7.7, “Sandbox”.

Here are some packages to help you to setup the virtualized system.

See Wikipedia article Comparison of platform virtual machines for detail comparison of different platform virtualization solutions.

	Note
	Default Debian kernels support KVM since lenny.

Typical work flow for virtualization involves several steps.

For the raw disk image file, see Section 9.7, “The disk image”.

For other virtual disk image files, you can use qemu-nbd(8) to export them using network block device protocol and mount them using the nbd kernel module.

qemu-nbd(8) supports disk formats supported by QEMU: QEMU supports following disk formats: raw, qcow2, qcow, vmdk, vdi, bochs, cow (user-mode Linux copy-on-write), parallels, dmg, cloop, vpc, vvfat (virtual VFAT), and host_device.

The network block device can support partitions in the same way as the loop device (see Section 9.7.3, “Mounting the disk image file”). You can mount the first partition of "disk.img" as follows.

# modprobe nbd max_part=16
# qemu-nbd -v -c /dev/nbd0 disk.img
...
# mkdir /mnt/part1
# mount /dev/nbd0p1 /mnt/part1

	Tip
	You may export only the first partition of "disk.img" using "-P 1" option to qemu-nbd(8).

If you wish to try a new Debian environment from a terminal console, I recommend you to use chroot. This enables you to run console applications of Debian unstable and testing without usual risks associated and without rebooting. chroot(8) is the most basic way.

	Caution
	Examples below assumes both parent system and chroot system share the same amd64 CPU architecture.

Although you can manually create a chroot(8) environment using debootstrap(1), this requires non-trivial efforts.

The sbuild package to build Debian packages from source uses the chroot environment managed by the schroot package. It comes with helper script sbuild-createchroot(1). Let's learn how it works by running it as follows.

$ sudo mkdir -p /srv/chroot
$ sudo sbuild-createchroot -v --include=eatmydata,ccache unstable /srv/chroot/unstable-amd64-sbuild http://deb.debian.org/debian
 ...

You see how debootstrap(8) populates system data for unstable environment under "/srv/chroot/unstable-amd64-sbuild" for a minimal build system.

You can login to this environment using schroot(1).

$ sudo schroot -v -c chroot:unstable-amd64-sbuild

You see how a system shell running under unstable environment is created.

	Note
	The "/usr/sbin/policy-rc.d" file which always exits with 101 prevents daemon programs to be started automatically on the Debian system. See "/usr/share/doc/init-system-helpers/README.policy-rc.d.gz".

	Note
	Some programs under chroot may require access to more files from the parent system to function than sbuild-createchroot provides as above. For example, "/sys", "/etc/passwd", "/etc/group", "/var/run/utmp", "/var/log/wtmp", etc. may need to be bind-mounted or copied.

	Tip
	The sbuild package helps to construct a chroot system and builds a package inside the chroot using schroot as its backend. It is an ideal system to check build-dependencies. See more on sbuild at Debian wiki and sbuild configuration example in "Guide for Debian Maintainers".

	Tip
	The systemd-nspawn(1) command helps to run a command or OS in a light-weight container in similar ways to chroot. It is more powerful since it uses namespaces to fully virtualize the the process tree, IPC, hostname, domain name and, optionally, networking and user databases. See systemd-nspawn.

If you wish to try a new GUI Desktop environment of any OS, I recommend you to use QEMU or KVM on a Debian stable system to run multiple desktop systems safely using virtualization. These enable you to run any desktop applications including ones of Debian unstable and testing without usual risks associated with them and without rebooting.

Since pure QEMU is very slow, it is recommended to accelerate it with KVM when the host system supports it.

Virtual Machine Manager also known as virt-manager is a convenient GUI tool for managing KVM virtual machines via libvirt.

The virtual disk image "virtdisk.qcow2" containing a Debian system for QEMU can be created using debian-installer: Small CDs as follows.

$ wget https://cdimage.debian.org/debian-cd/5.0.3/amd64/iso-cd/debian-503-amd64-netinst.iso
$ qemu-img create -f qcow2 virtdisk.qcow2 5G
$ qemu -hda virtdisk.qcow2 -cdrom debian-503-amd64-netinst.iso -boot d -m 256
...

	Tip
	Running other GNU/Linux distributions such as Ubuntu and Fedora under virtualization is a great way to learn configuration tips. Other proprietary OSs may be run nicely under this GNU/Linux virtualization, too.

See more tips at Debian wiki: SystemVirtualization.