7

Say I'm running some lengthy process, consisting of two steps: a generates a big intermediate file, say /tmp/aoutput, then b reads that file and does something further with it.

Now the system crashed during the b step. Is it possible to reboot the system in a way that will not delete the contents of /tmp, as would usually happen on boot, so that we can immediately restart b without first running a completely?

This assumes of course that /tmp is not mounted in tmpfs.

I'd like to preclude solutions that involve physically taking out the HDD/SSD or booting with a live USB medium. In my concrete situation now, it's actually an SSD with full disk encryption, but I'd also be interested in solutions that only work with a non-encrypted drive.

edit Still keeping the question open for now, because neither of the answers gives a solution that works without either a live medium or tweaking the configuration beforehand. Maybe it's really not possible any other way, but it seems odd: doesn't GRUB already mount partitions, in order to boot Linux? If so, isn't there a way to manually move the files, before ever launching systemd? Perhaps I'm wrong, I know little about bootloaders; but then I'd like to hear the reason why this is impossible.

systemd configuration info:

$ grep -s . /*/tmpfiles.d/*.conf /*/*/tmpfiles.d/*.conf
/run/tmpfiles.d/kmod.conf:c! /dev/fuse 0600 - - - 10:229
/run/tmpfiles.d/kmod.conf:c! /dev/cuse 0600 - - - 10:203
/run/tmpfiles.d/kmod.conf:c! /dev/btrfs-control 0600 - - - 10:234
/run/tmpfiles.d/kmod.conf:d /dev/net 0755 - - -
/run/tmpfiles.d/kmod.conf:c! /dev/net/tun 0600 - - - 10:200
/run/tmpfiles.d/kmod.conf:c! /dev/ppp 0600 - - - 108:0
/run/tmpfiles.d/kmod.conf:c! /dev/uinput 0600 - - - 10:223
/run/tmpfiles.d/kmod.conf:d /dev/mapper 0755 - - -
/run/tmpfiles.d/kmod.conf:c! /dev/mapper/control 0600 - - - 10:236
/run/tmpfiles.d/kmod.conf:c! /dev/uhid 0600 - - - 10:239
/run/tmpfiles.d/kmod.conf:c! /dev/vhci 0600 - - - 10:137
/run/tmpfiles.d/kmod.conf:d /dev/snd 0755 - - -
/run/tmpfiles.d/kmod.conf:c! /dev/snd/timer 0600 - - - 116:33
/run/tmpfiles.d/kmod.conf:d /dev/snd 0755 - - -
/run/tmpfiles.d/kmod.conf:c! /dev/snd/seq 0600 - - - 116:1
/usr/lib/tmpfiles.d/00rsyslog.conf:# Override systemd's default tmpfiles.d/var.conf to make /var/log writable by
/usr/lib/tmpfiles.d/00rsyslog.conf:# the syslog group, so that rsyslog can run as user.
/usr/lib/tmpfiles.d/00rsyslog.conf:# See tmpfiles.d(5) for details.
/usr/lib/tmpfiles.d/00rsyslog.conf:# Type Path    Mode UID  GID  Age Argument
/usr/lib/tmpfiles.d/00rsyslog.conf:d /var/log 0775 root syslog -
/usr/lib/tmpfiles.d/dbus.conf:# Type Path                     Mode    UID     GID     Age     Argument
/usr/lib/tmpfiles.d/dbus.conf:L /var/lib/dbus/machine-id      -       -       -       -       /etc/machine-id
/usr/lib/tmpfiles.d/debian.conf:#  This file is part of the debianisation of systemd.
/usr/lib/tmpfiles.d/debian.conf:#
/usr/lib/tmpfiles.d/debian.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/debian.conf:#  under the terms of the GNU General Public License as published by
/usr/lib/tmpfiles.d/debian.conf:#  the Free Software Foundation; either version 2 of the License, or
/usr/lib/tmpfiles.d/debian.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/debian.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/debian.conf:# Type Path    Mode UID  GID  Age Argument
/usr/lib/tmpfiles.d/debian.conf:L /run/initctl -    -    -    -   /run/systemd/initctl/fifo
/usr/lib/tmpfiles.d/debian.conf:L /run/shm     -    -    -    -   /dev/shm
/usr/lib/tmpfiles.d/debian.conf:d /run/sendsigs.omit.d 0755 root root -
/usr/lib/tmpfiles.d/debian.conf:L+ /etc/mtab   -    -    -    -  ../proc/self/mounts
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# This is a systemd tmpfiles.d configuration file
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:#
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# tmpfiles.d defaults are set to clean /run/user every now and then
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# which includes our gvfs-fuse mount being mounted in /run/user/<id>/gvfs
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:#
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# This file adds an exclusion rule so that user data don't get automatically
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# cleaned up (i.e. destroyed).
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:#
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# Due to our fuse mount restrictions root can't access nor stat the mountpoint
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# resulting in warning spitted out by the systemd-tmpfiles process. Please
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# ignore it for the time being until proper solution is found:
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:# https://bugzilla.gnome.org/show_bug.cgi?id=560658
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf:x /run/user/*/gvfs
/usr/lib/tmpfiles.d/home.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/home.conf:#
/usr/lib/tmpfiles.d/home.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/home.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/home.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/home.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/home.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/home.conf:Q /home 0755 - - -
/usr/lib/tmpfiles.d/home.conf:q /srv 0755 - - -
/usr/lib/tmpfiles.d/journal-nocow.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/journal-nocow.conf:#
/usr/lib/tmpfiles.d/journal-nocow.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/journal-nocow.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/journal-nocow.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/journal-nocow.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/journal-nocow.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/journal-nocow.conf:# Set the NOCOW attribute for directories of journal files. This flag
/usr/lib/tmpfiles.d/journal-nocow.conf:# is inheredited by their new files and sub-directories. Matters only
/usr/lib/tmpfiles.d/journal-nocow.conf:# for btrfs filesystems.
/usr/lib/tmpfiles.d/journal-nocow.conf:#
/usr/lib/tmpfiles.d/journal-nocow.conf:# WARNING: Enabling the NOCOW attribute improves journal performance
/usr/lib/tmpfiles.d/journal-nocow.conf:#     substantially, but also disables the btrfs checksum logic. In
/usr/lib/tmpfiles.d/journal-nocow.conf:#     btrfs RAID filesystems the checksums are needed for rebuilding
/usr/lib/tmpfiles.d/journal-nocow.conf:#     corrupted files. Without checksums such rebuilds are not
/usr/lib/tmpfiles.d/journal-nocow.conf:#     possible.
/usr/lib/tmpfiles.d/journal-nocow.conf:#
/usr/lib/tmpfiles.d/journal-nocow.conf:# In a single-disk filesystem (or a filesystem without redundancy)
/usr/lib/tmpfiles.d/journal-nocow.conf:# enabling the NOCOW attribute for journal files is safe, because
/usr/lib/tmpfiles.d/journal-nocow.conf:# they have their own checksums and a rebuilding wouldn't be possible
/usr/lib/tmpfiles.d/journal-nocow.conf:# in any case.
/usr/lib/tmpfiles.d/journal-nocow.conf:h /var/log/journal - - - - +C
/usr/lib/tmpfiles.d/journal-nocow.conf:h /var/log/journal/%m - - - - +C
/usr/lib/tmpfiles.d/journal-nocow.conf:h /var/log/journal/remote - - - - +C
/usr/lib/tmpfiles.d/legacy.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/legacy.conf:#
/usr/lib/tmpfiles.d/legacy.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/legacy.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/legacy.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/legacy.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/legacy.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/legacy.conf:# These files are considered legacy and are unnecessary on legacy-free
/usr/lib/tmpfiles.d/legacy.conf:# systems.
/usr/lib/tmpfiles.d/legacy.conf:L /var/lock - - - - ../run/lock
/usr/lib/tmpfiles.d/legacy.conf:# /run/lock/subsys is used for serializing SysV service execution, and
/usr/lib/tmpfiles.d/legacy.conf:# hence without use on SysV-less systems.
/usr/lib/tmpfiles.d/legacy.conf:d /run/lock/subsys 0755 root root -
/usr/lib/tmpfiles.d/legacy.conf:# /forcefsck, /fastboot and /forcequotacheck are deprecated in favor of the
/usr/lib/tmpfiles.d/legacy.conf:# kernel command line options 'fsck.mode=force', 'fsck.mode=skip' and
/usr/lib/tmpfiles.d/legacy.conf:# 'quotacheck.mode=force'
/usr/lib/tmpfiles.d/legacy.conf:r! /forcefsck
/usr/lib/tmpfiles.d/legacy.conf:r! /fastboot
/usr/lib/tmpfiles.d/legacy.conf:r! /forcequotacheck
/usr/lib/tmpfiles.d/man-db.conf:d /var/cache/man 2755 man root 1w
/usr/lib/tmpfiles.d/passwd.conf:# If a password operation is in progress and we lose power, stale lockfiles
/usr/lib/tmpfiles.d/passwd.conf:# can be left behind.  Clear them on boot.
/usr/lib/tmpfiles.d/passwd.conf:r! /etc/gshadow.lock
/usr/lib/tmpfiles.d/passwd.conf:r! /etc/shadow.lock
/usr/lib/tmpfiles.d/passwd.conf:r! /etc/passwd.lock
/usr/lib/tmpfiles.d/passwd.conf:r! /etc/group.lock
/usr/lib/tmpfiles.d/passwd.conf:r! /etc/subuid.lock
/usr/lib/tmpfiles.d/passwd.conf:r! /etc/subgid.lock
/usr/lib/tmpfiles.d/samba.conf:d /run/samba 0755 root root -
/usr/lib/tmpfiles.d/sshd.conf:d /var/run/sshd 0755 root root
/usr/lib/tmpfiles.d/sudo.conf:# Create an empty sudo time stamp directory on OSes using systemd.
/usr/lib/tmpfiles.d/sudo.conf:# Sudo will create the directory itself but this can cause problems
/usr/lib/tmpfiles.d/sudo.conf:# on systems that have SELinux enabled since the directories will be
/usr/lib/tmpfiles.d/sudo.conf:# created with the user's security context.
/usr/lib/tmpfiles.d/sudo.conf:d /var/run/sudo 0711 root root
/usr/lib/tmpfiles.d/sudo.conf:D /var/run/sudo/ts 0700 root root
/usr/lib/tmpfiles.d/systemd.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/systemd.conf:#
/usr/lib/tmpfiles.d/systemd.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/systemd.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/systemd.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/systemd.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/systemd.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/systemd.conf:d /run/user 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:F! /run/utmp 0664 root utmp -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/ask-password 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/seats 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/sessions 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/users 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/machines 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/shutdown 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/netif 0755 systemd-network systemd-network -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/netif/links 0755 systemd-network systemd-network -
/usr/lib/tmpfiles.d/systemd.conf:d /run/systemd/netif/leases 0755 systemd-network systemd-network -
/usr/lib/tmpfiles.d/systemd.conf:d /run/log 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:z /run/log/journal 2755 root systemd-journal - -
/usr/lib/tmpfiles.d/systemd.conf:Z /run/log/journal/%m ~2750 root systemd-journal - -
/usr/lib/tmpfiles.d/systemd.conf:a+ /run/log/journal/%m - - - - d:group:adm:r-x
/usr/lib/tmpfiles.d/systemd.conf:A+ /run/log/journal/%m - - - - group:adm:r-x
/usr/lib/tmpfiles.d/systemd.conf:z /var/log/journal 2755 root systemd-journal - -
/usr/lib/tmpfiles.d/systemd.conf:z /var/log/journal/%m 2755 root systemd-journal - -
/usr/lib/tmpfiles.d/systemd.conf:z /var/log/journal/%m/system.journal 0640 root systemd-journal - -
/usr/lib/tmpfiles.d/systemd.conf:a+ /var/log/journal    - - - - d:group:adm:r-x
/usr/lib/tmpfiles.d/systemd.conf:a+ /var/log/journal    - - - - group:adm:r-x
/usr/lib/tmpfiles.d/systemd.conf:a+ /var/log/journal/%m - - - - d:group:adm:r-x
/usr/lib/tmpfiles.d/systemd.conf:a+ /var/log/journal/%m - - - - group:adm:r-x
/usr/lib/tmpfiles.d/systemd.conf:a+ /var/log/journal/%m/system.journal - - - - group:adm:r--
/usr/lib/tmpfiles.d/systemd.conf:d /var/lib/systemd 0755 root root -
/usr/lib/tmpfiles.d/systemd.conf:d /var/lib/systemd/coredump 0755 root root 3d
/usr/lib/tmpfiles.d/systemd-nologin.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/systemd-nologin.conf:#
/usr/lib/tmpfiles.d/systemd-nologin.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/systemd-nologin.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/systemd-nologin.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/systemd-nologin.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/systemd-nologin.conf:# See tmpfiles.d(5), systemd-user-session.service(5) and pam_nologin(8).
/usr/lib/tmpfiles.d/systemd-nologin.conf:# This file has special suffix so it is not run by mistake.
/usr/lib/tmpfiles.d/systemd-nologin.conf:F! /run/nologin 0644 - - - "System is booting up. See pam_nologin(8)"
/usr/lib/tmpfiles.d/tmp.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/tmp.conf:#
/usr/lib/tmpfiles.d/tmp.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/tmp.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/tmp.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/tmp.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/tmp.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/tmp.conf:# Clear tmp directories separately, to make them easier to override
/usr/lib/tmpfiles.d/tmp.conf:D /tmp 1777 root root -
/usr/lib/tmpfiles.d/tmp.conf:#q /var/tmp 1777 root root 30d
/usr/lib/tmpfiles.d/tmp.conf:# Exclude namespace mountpoints created with PrivateTmp=yes
/usr/lib/tmpfiles.d/tmp.conf:x /tmp/systemd-private-%b-*
/usr/lib/tmpfiles.d/tmp.conf:X /tmp/systemd-private-%b-*/tmp
/usr/lib/tmpfiles.d/tmp.conf:x /var/tmp/systemd-private-%b-*
/usr/lib/tmpfiles.d/tmp.conf:X /var/tmp/systemd-private-%b-*/tmp
/usr/lib/tmpfiles.d/var.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/var.conf:#
/usr/lib/tmpfiles.d/var.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/var.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/var.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/var.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/var.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/var.conf:q /var 0755 - - -
/usr/lib/tmpfiles.d/var.conf:L /var/run - - - - ../run
/usr/lib/tmpfiles.d/var.conf:d /var/log 0755 - - -
/usr/lib/tmpfiles.d/var.conf:f /var/log/wtmp 0664 root utmp -
/usr/lib/tmpfiles.d/var.conf:f /var/log/btmp 0600 root utmp -
/usr/lib/tmpfiles.d/var.conf:d /var/cache 0755 - - -
/usr/lib/tmpfiles.d/var.conf:d /var/lib 0755 - - -
/usr/lib/tmpfiles.d/var.conf:d /var/spool 0755 - - -
/usr/lib/tmpfiles.d/x11.conf:#  This file is part of systemd.
/usr/lib/tmpfiles.d/x11.conf:#
/usr/lib/tmpfiles.d/x11.conf:#  systemd is free software; you can redistribute it and/or modify it
/usr/lib/tmpfiles.d/x11.conf:#  under the terms of the GNU Lesser General Public License as published by
/usr/lib/tmpfiles.d/x11.conf:#  the Free Software Foundation; either version 2.1 of the License, or
/usr/lib/tmpfiles.d/x11.conf:#  (at your option) any later version.
/usr/lib/tmpfiles.d/x11.conf:# See tmpfiles.d(5) for details
/usr/lib/tmpfiles.d/x11.conf:# Make sure these are created by default so that nobody else can
/usr/lib/tmpfiles.d/x11.conf:d /tmp/.X11-unix 1777 root root 10d
/usr/lib/tmpfiles.d/x11.conf:d /tmp/.ICE-unix 1777 root root 10d
/usr/lib/tmpfiles.d/x11.conf:d /tmp/.XIM-unix 1777 root root 10d
/usr/lib/tmpfiles.d/x11.conf:d /tmp/.font-unix 1777 root root 10d
/usr/lib/tmpfiles.d/x11.conf:d /tmp/.Test-unix 1777 root root 10d
/usr/lib/tmpfiles.d/x11.conf:# Unlink the X11 lock files
/usr/lib/tmpfiles.d/x11.conf:r! /tmp/.X[0-9]*-lock
/var/run/tmpfiles.d/kmod.conf:c! /dev/fuse 0600 - - - 10:229
/var/run/tmpfiles.d/kmod.conf:c! /dev/cuse 0600 - - - 10:203
/var/run/tmpfiles.d/kmod.conf:c! /dev/btrfs-control 0600 - - - 10:234
/var/run/tmpfiles.d/kmod.conf:d /dev/net 0755 - - -
/var/run/tmpfiles.d/kmod.conf:c! /dev/net/tun 0600 - - - 10:200
/var/run/tmpfiles.d/kmod.conf:c! /dev/ppp 0600 - - - 108:0
/var/run/tmpfiles.d/kmod.conf:c! /dev/uinput 0600 - - - 10:223
/var/run/tmpfiles.d/kmod.conf:d /dev/mapper 0755 - - -
/var/run/tmpfiles.d/kmod.conf:c! /dev/mapper/control 0600 - - - 10:236
/var/run/tmpfiles.d/kmod.conf:c! /dev/uhid 0600 - - - 10:239
/var/run/tmpfiles.d/kmod.conf:c! /dev/vhci 0600 - - - 10:137
/var/run/tmpfiles.d/kmod.conf:d /dev/snd 0755 - - -
/var/run/tmpfiles.d/kmod.conf:c! /dev/snd/timer 0600 - - - 116:33
/var/run/tmpfiles.d/kmod.conf:d /dev/snd 0755 - - -
/var/run/tmpfiles.d/kmod.conf:c! /dev/snd/seq 0600 - - - 116:1
Improve this question
5
  • Re your edit, the boot loader can read file systems, it can’t (AFAIK) make changes to them. However I think it should be possible to disable units using kernel command-line options, so there might be another way. Commented Sep 3, 2020 at 9:49
  • 1
    Part of the problem here is that files in /tmp are not supposed to survive past a reboot as per the file system standard. Generally, if you want file to survive past reboot they should be written to /var/tmp instead. I understand that in this case the system was not supposed to be rebooted either. Commented Sep 3, 2020 at 10:01
  • 2
    @Philip I’d say “guaranteed” rather than “supposed” — the FHS allows /tmp to be preserved or cleared across reboots. As you say though, /var/tmp is better for this... (But that’s not relevant in this particular case, which is about dealing with an existing situation, not best practice.) Commented Sep 3, 2020 at 10:05
  • If /tmp is on the root file system then I wonder if passing in ro as a boot param would work. You can edit boot params in grub before booting. I'm not sure if systemd automatically remounts this though. Commented Sep 3, 2020 at 10:06
  • GRUB only knows some file systems and only mounts them read-only. To move/copy those files, you need write-access. That's only provided by initramfs. But at that point, pid1/systemd is already running. I guess the easiest way would be to prepare a RO rescue system on the machine itself, e.g. a GRML squashfs/cpio for GRUB to boot. If you prepare the rescue systems /etc/crypttab and /etc/fstab then it will ask you for the password and you can then copy the files wherever you want. But I haven't checked that. Commented Sep 4, 2020 at 8:41

5 Answers 5

Reset to default
5

TL;DR: Use a live medium if the crash just happened. Otherwise configure your system by copying /usr/lib/tmpfiles.d/tmp.conf to /etc/tmpfiles.d/tmp.conf and change the file in etc:

-D /tmp 1777 root root -
+d /tmp 1777 root root 1s
+x /tmp/*output

If your files don't follow a specific pattern, then use the following settings instead:

-D /tmp 1777 root root -
+d /tmp 1777 root root 12h

Now the system crashed during the b step. Is it possible to reboot the system in a way that will not delete the contents of /tmp, as would usually happen on boot, so that we can immediately restart b without first running a completely?

Well, somewhat, if you can configure the system before the crash happens. If the system is already crashed at this very moment, then you must use a live medium, as every usual systemd.target (rescue, multi-user, graphical) includes the deletion of temporary files, even the rescue.target.

That being said, to be ready for the next crash, we can change your tmpfiles behavior. Most Debian-derived systems define the behaviour for /tmp in /usr/lib/tmpfiles.d/tmp.conf:

$ cat /usr/lib/tmpfiles.d/tmp.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# See tmpfiles.d(5) for details

# Clear tmp directories separately, to make them easier to override
D /tmp 1777 root root -
#q /var/tmp 1777 root root 30d

# Exclude namespace mountpoints created with PrivateTmp=yes
x /tmp/systemd-private-%b-*
X /tmp/systemd-private-%b-*/tmp
x /var/tmp/systemd-private-%b-*
X /var/tmp/systemd-private-%b-*/tmp

# Remove top-level private temporary directories on each boot
R! /tmp/systemd-private-*
R! /var/tmp/systemd-private-*

The line responsible for the cleanup is

D /tmp 1777 root root -

As per tmpfiles.d(5), a capital D indicates that the directory should get removed on systemd-tmpfiles --remove. Unfortunately that command runs on almost every target due to systemd-tmpfiles-setup.service:

$ systemctl cat systemd-tmpfiles-setup.service | grep Exec
ExecStart=/bin/systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev

$ systemctl list-dependencies --reverse systemd-tmpfiles-setup.service | grep target
● └─sysinit.target
●   ├─basic.target
●   │ └─multi-user.target
●   │   └─graphical.target
●   └─rescue.target

Also, there are several timers that trigger a --clean operation and thus also remove old files.

Depending on your scenario, you have several solutions at hand, none of which are perfect.

Warning: Regardless of what you do, do not change /usr/lib/tmpfiles.d/'s content. Instead, create a file with the same name in /etc/tmpfiles.d, e.g.

sudo cp /usr/lib/tmpfiles.d/tmp.conf /etc/tmpfiles.d/tmp.conf

tmpfiles.d follows the usual systemd rules: /etc > /run > /usr/lib, i.e. /etc/ gets priority.

Option 1: Change the service permanently

/tmp gets removed due to --remove in the aforementioned service. You can systemctl edit the ExecStart argument to remove the --remove parameter:

$ systemctl edit systemd-tmpfiles-setup.service
ExecStart=
ExecStart=/bin/systemd-tmpfiles --create --boot --exclude-prefix=/dev

The additional empty ExecStart is necessary to remove the original configuration. Alternatively run systemctl edit --full ... and edit the resulting file.

As --remove is missing, D in all your configuration files is effectively disabled, which might not be helpful and thus discouraged by me. So instead you probably want to change the definition of your /tmp rule:

Option 2: Clean /tmp only for files of specific age

Copy /usr/lib/tmpfiles.d/tmp.conf to /etc/tmpfiles.d/tmp.conf and replace the D line

-D /tmp 1777 root root -
+d /tmp 1777 root root 12h

This will only delete files that are older than 12 hours instead of all files. If you reboot immediately after a crash, then this might be enough for you. However, we can even go a step further.

Option 3: Exclude project files

If your files always have the same name/pattern, we can prevent them from being deleted with an exclusion:

-D /tmp 1777 root root -
+d /tmp 1777 root root 1s
+x /tmp/*output

Note that 1s will effectively remove all files when systemd-tmpfiles is called, except /tmp/*output files, e.g. /tmp/aoutput. You can also exclude whole folders.

The last variant should be sufficient for your problem and prevent you from getting a /tmp/ that's too large.

Checking the configuration

After you've changed the configuration, it's time to test it. This test is the same, regardless which variant you've used. Create files that have similar names in your /tmp and then run the systemd services:

$ touch /tmp/my-project-files-{a,b,c,d,e,f} /tmp/some-test-garbage{1..10}
$ systemctl start systemd-tmpfiles-{clean,setup}.service

If everything is set up correctly, your "project" files should still be there. If you've used the exclusion variant, then the other garbage in /tmp/ should be gone.

References

  • tmpfiles.d(5) for the syntax in tmpfiles.d/* and the precedence of the files
  • systemd.service(5) for ExecStart
  • systemd.unit(5) for the ExecStart= override
  • systemd-tmpfiles(8) for an overview of the tmpfiles landscape
  • the systemd-tmpfiles-*.service files via systemctl cat systemd-tmpfiles-{clean,setup,setup-dev}.service
Improve this answer
2
  • Thanks for a very extensive answer! — I'm not really happy with permanently changing the configuration; after all, the reason I put these files in /tmp in the first place is that normally I want them to be deleted on reboot – provided I shut down the system myself. So, I take it it's not possible to either a) somehow make the deletion dependent on whether the shutdown was orderly or REISUB b) manually prevent deletion from Grub? Commented Sep 3, 2020 at 9:51
  • @leftaroundabout ad a) I don't know any systemd facility for that, sorry. There might be something, but I'm not aware of anything. ad b): Unfortunately not. systemd-tmpfiles-setup.service does not only --remove, but also --create, and the creation of the additional directories is vital, as several other services need their specific directories. As such, the systemd-tmpfiles-setup.service is loaded by all systemd.unit=... targets (at least on my quick skim of Debian's configuration). Commented Sep 4, 2020 at 8:34
3

If you just want to salvage some files under /tmp you can follow these steps:

  1. Enter grub edit mode by pressing "e" in a grub menu entry
  2. Add "break=mount" at the end of the line that starts with "linux"
  3. Press F10 to boot and you should end up in an (initramfs) console
  4. Run blkid to get en overview over available filesystems
  5. Run mkdir /mnt, then modprobe ext4, and something like mount /dev/nvme0n1p1 /mnt, to mount the partition of your interest at /mnt
  6. Move the files of your interest to a permanent location with something like mv /mnt/tmp/important_file /mnt/home/myuser/
  7. Reboot with reboot -f and start your system normally
Improve this answer
0
2

You can add init=/bin/bash to the kernel command line to boot to a shell without starting systemd. Remount the filesystem read-write, move the files out of /tmp, mount the filesystem read-only again, and exec /sbin/init to boot the system.

Note that the shell won't have a fully set up terminal, so some features (tab completion, control-Z, control-C,...) might not work.

I suggest trying this on a test system first, to make sure it's working for you without risking your file.

PS: normally /var/tmp is preserved across boots.

Improve this answer
1
  • This is at least a variant which may work out-of-the box and does not require prior preparation. Commented Mar 28, 2024 at 19:38
2

In case /tmp is some kind of memory baked file system you are obviously out of luck. Otherwise it's easy to accomplish – at least if your distribution uses systemd.

On a typical system using systemd, there are dedicated services named systemd-tmpfiles-clean.service and systemd-tmpfiles-setup.service (please refer to man systemd-tmpfiles) which accomplish to clean up temporary files. You are able to mask these specific service e.g. using the following kernel parameters (please refer to man kernel-command-line) for the final system, not the initrd, assuming the service is only used there:

systemd.mask=systemd-tmpfiles-clean.service systemd.mask=systemd-tmpfiles-setup.service

How to change the kernel parameters depends on used boot loader. In case of Grub it's as easy as press e within its menu and append it to the linux command.

Be aware that a minor typo will may be catastrophic in this case.

Keep also in mind that you want to unmask these services at some point in time. systemctl unmask <unit> seems not to work, hence the simplest way I am aware of is to reboot.

Tested successfully on a recent Debian.

Improve this answer
0
0

Yes it is.

There are two things that can result in lost files in /tmp:

  1. tmpfs: Many modern systems mount a temporary file system (think RAM disk) in /tmp which means the contents are always lost at shutdown.

  2. Cleaning of old files in /tmp. This is a configurable script to clean files in /tmp.

Please check the output of the command df /tmp. In case 1 it will look similar to this:

$ df /tmp/
Filesystem     1K-blocks  Used Available Use% Mounted on
tmpfs           12290040   276  12289764   1% /tmp

(Important is the first column. tmpfs.)

In case 2 (your case) you can often find out more by looking at the files in /etc/tmpfiles.d -- see man tmpfiles.d. Please add the output of

grep -s . /*/tmpfiles.d/*.conf /*/*/tmpfiles.d/*.conf

to your question. ⟹ Your file is /usr/lib/tmpfiles.d/tmp.conf which you need to edit, e.g. with sudo nano /usr/lib/tmpfiles.d/tmp.conf and apply the change according to the following example.

This is an example file /etc/tmpfiles.d/tmp.conf:

# see tmpfiles.d(5)
# always enable /tmp folder cleaning
D! /tmp 1777 root root 0

# remove files in /var/tmp older than 10 days
D /var/tmp 1777 root root 10d

The first entry D! /tmp 1777 root root 0 means that the /tmp folder will be removed and re-created at each boot. You could comment out that line and perhaps add a line similar to the next non-comment line in that file:

# see tmpfiles.d(5)

# # always enable /tmp folder cleaning
# D! /tmp 1777 root root 0

# Keep files in /tmp but not very old ones
D! /var/tmp 1777 root root 30d

# remove files in /var/tmp older than 10 days
D /var/tmp 1777 root root 10d

As the system is not booted right now, you need to boot from a live DVD/USB medium, then mount your system partition, e.g. to /mnt. Then edit the file /mnt/usr/lib/tmpfiles.d/tmp.conf as described above.

Improve this answer
11
  • The question specifies that it’s not a tmpfs. Commented Sep 2, 2020 at 12:48
  • @StephenKitt OK, but the answer should perhaps stay that way to be generic. Will edit and add now. Commented Sep 2, 2020 at 12:49
  • There's actually nothing in /etc/tmpfiles.d, but something very similar to your example in /usr/lib/tmpfiles.d. So, that file would need to edited from Grub before booting Linux? Commented Sep 2, 2020 at 13:17
  • @leftaroundabout Yes, that would probably be the file. Please add the output of grep -s . /*/tmpfiles.d/*.conf /*/*/tmpfiles.d/*.conf to your question (as code block) by editing it. I will add information in my answer then. Commented Sep 2, 2020 at 15:02
  • That's a lot, but ok. But hopefully it's possible to answer in a way that's not totally specific to my particular system configuration? Commented Sep 2, 2020 at 15:10

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.