File manager

File manager - Edit - /usr/share/doc/restic/html/045_working_with_repos.html

Back
<!DOCTYPE html> <html class="writer-html5" lang="en" > <head> <meta charset="utf-8" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Working with repositories — restic 0.12.1 documentation</title> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="_static/css/restic.css" type="text/css" /> <link rel="shortcut icon" href="_static/favicon.ico"/> <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script> <script src="_static/jquery.js"></script> <script src="_static/underscore.js"></script> <script src="_static/doctools.js"></script> <script src="_static/js/theme.js"></script> <link rel="index" title="Index" href="genindex.html" /> <link rel="search" title="Search" href="search.html" /> <link rel="next" title="Restoring from backup" href="050_restore.html" /> <link rel="prev" title="Backing up" href="040_backup.html" /> </head> <body class="wy-body-for-nav"> <div class="wy-grid-for-nav"> <nav data-toggle="wy-nav-shift" class="wy-nav-side"> <div class="wy-side-scroll"> <div class="wy-side-nav-search" > <a href="index.html" class="icon icon-home"> restic <img src="_static/logo.png" class="logo" alt="Logo"/> </a> <div class="version"> 0.12.1 </div> <div role="search"> <form id="rtd-search-form" class="wy-form" action="search.html" method="get"> <input type="text" name="q" placeholder="Search docs" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu"> <ul class="current"> <li class="toctree-l1"><a class="reference internal" href="010_introduction.html">Introduction</a></li> <li class="toctree-l1"><a class="reference internal" href="020_installation.html">Installation</a></li> <li class="toctree-l1"><a class="reference internal" href="030_preparing_a_new_repo.html">Preparing a new repository</a></li> <li class="toctree-l1"><a class="reference internal" href="040_backup.html">Backing up</a></li> <li class="toctree-l1 current"><a class="current reference internal" href="#">Working with repositories</a><ul> <li class="toctree-l2"><a class="reference internal" href="#listing-all-snapshots">Listing all snapshots</a></li> <li class="toctree-l2"><a class="reference internal" href="#copying-snapshots-between-repositories">Copying snapshots between repositories</a><ul> <li class="toctree-l3"><a class="reference internal" href="#filtering-snapshots-to-copy">Filtering snapshots to copy</a></li> <li class="toctree-l3"><a class="reference internal" href="#ensuring-deduplication-for-copied-snapshots">Ensuring deduplication for copied snapshots</a></li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="#checking-integrity-and-consistency">Checking integrity and consistency</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="050_restore.html">Restoring from backup</a></li> <li class="toctree-l1"><a class="reference internal" href="060_forget.html">Removing backup snapshots</a></li> <li class="toctree-l1"><a class="reference internal" href="070_encryption.html">Encryption</a></li> <li class="toctree-l1"><a class="reference internal" href="075_scripting.html">Scripting</a></li> <li class="toctree-l1"><a class="reference internal" href="080_examples.html">Examples</a></li> <li class="toctree-l1"><a class="reference internal" href="090_participating.html">Participating</a></li> <li class="toctree-l1"><a class="reference internal" href="100_references.html">References</a></li> <li class="toctree-l1"><a class="reference internal" href="110_talks.html">Talks</a></li> <li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li> <li class="toctree-l1"><a class="reference internal" href="manual_rest.html">Manual</a></li> <li class="toctree-l1"><a class="reference internal" href="developer_information.html">Developer Information</a></li> </ul> </div> </div> </nav> <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" > <i data-toggle="wy-nav-top" class="fa fa-bars"></i> <a href="index.html">restic</a> </nav> <div class="wy-nav-content"> <div class="rst-content"> <div role="navigation" aria-label="Page navigation"> <ul class="wy-breadcrumbs"> <li><a href="index.html" class="icon icon-home"></a> »</li> <li>Working with repositories</li> <li class="wy-breadcrumbs-aside"> <a href="_sources/045_working_with_repos.rst.txt" rel="nofollow"> View page source</a> </li> </ul> <hr/> </div> <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> <div itemprop="articleBody"> <section id="working-with-repositories"> <h1>Working with repositories<a class="headerlink" href="#working-with-repositories" title="Permalink to this headline">¶</a></h1> <section id="listing-all-snapshots"> <h2>Listing all snapshots<a class="headerlink" href="#listing-all-snapshots" title="Permalink to this headline">¶</a></h2> <p>Now, you can list all the snapshots stored in the repository:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo snapshots <span class="go">enter password for repository:</span> <span class="go">ID Date Host Tags Directory</span> <span class="go">----------------------------------------------------------------------</span> <span class="go">40dc1520 2015-05-08 21:38:30 kasimir /home/user/work</span> <span class="go">79766175 2015-05-08 21:40:19 kasimir /home/user/work</span> <span class="go">bdbd3439 2015-05-08 21:45:17 luigi /home/art</span> <span class="go">590c8fc8 2015-05-08 21:47:38 kazik /srv</span> <span class="go">9f0bc19e 2015-05-08 21:46:11 luigi /srv</span> </pre></div> </div> <p>You can filter the listing by directory path:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo snapshots --path<span class="o">=</span><span class="s2">"/srv"</span> <span class="go">enter password for repository:</span> <span class="go">ID Date Host Tags Directory</span> <span class="go">----------------------------------------------------------------------</span> <span class="go">590c8fc8 2015-05-08 21:47:38 kazik /srv</span> <span class="go">9f0bc19e 2015-05-08 21:46:11 luigi /srv</span> </pre></div> </div> <p>Or filter by host:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo snapshots --host luigi <span class="go">enter password for repository:</span> <span class="go">ID Date Host Tags Directory</span> <span class="go">----------------------------------------------------------------------</span> <span class="go">bdbd3439 2015-05-08 21:45:17 luigi /home/art</span> <span class="go">9f0bc19e 2015-05-08 21:46:11 luigi /srv</span> </pre></div> </div> <p>Combining filters is also possible.</p> <p>Furthermore you can group the output by the same filters (host, paths, tags):</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo snapshots --group-by host <span class="go">enter password for repository:</span> <span class="go">snapshots for (host [kasimir])</span> <span class="go">ID Date Host Tags Directory</span> <span class="go">----------------------------------------------------------------------</span> <span class="go">40dc1520 2015-05-08 21:38:30 kasimir /home/user/work</span> <span class="go">79766175 2015-05-08 21:40:19 kasimir /home/user/work</span> <span class="go">2 snapshots</span> <span class="go">snapshots for (host [luigi])</span> <span class="go">ID Date Host Tags Directory</span> <span class="go">----------------------------------------------------------------------</span> <span class="go">bdbd3439 2015-05-08 21:45:17 luigi /home/art</span> <span class="go">9f0bc19e 2015-05-08 21:46:11 luigi /srv</span> <span class="go">2 snapshots</span> <span class="go">snapshots for (host [kazik])</span> <span class="go">ID Date Host Tags Directory</span> <span class="go">----------------------------------------------------------------------</span> <span class="go">590c8fc8 2015-05-08 21:47:38 kazik /srv</span> <span class="go">1 snapshots</span> </pre></div> </div> </section> <section id="copying-snapshots-between-repositories"> <h2>Copying snapshots between repositories<a class="headerlink" href="#copying-snapshots-between-repositories" title="Permalink to this headline">¶</a></h2> <p>In case you want to transfer snapshots between two repositories, for example from a local to a remote repository, you can use the <code class="docutils literal notranslate"><span class="pre">copy</span></code> command:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo copy --repo2 /srv/restic-repo-copy <span class="go">repository d6504c63 opened successfully, password is correct</span> <span class="go">repository 3dd0878c opened successfully, password is correct</span> <span class="go">snapshot 410b18a2 of [/home/user/work] at 2020-06-09 23:15:57.305305 +0200 CEST)</span> <span class="go"> copy started, this may take a while...</span> <span class="go">snapshot 7a746a07 saved</span> <span class="go">snapshot 4e5d5487 of [/home/user/work] at 2020-05-01 22:44:07.012113 +0200 CEST)</span> <span class="go">skipping snapshot 4e5d5487, was already copied to snapshot 50eb62b7</span> </pre></div> </div> <p>The example command copies all snapshots from the source repository <code class="docutils literal notranslate"><span class="pre">/srv/restic-repo</span></code> to the destination repository <code class="docutils literal notranslate"><span class="pre">/srv/restic-repo-copy</span></code>. Snapshots which have previously been copied between repositories will be skipped by later copy runs.</p> <div class="admonition important"> <p class="admonition-title">Important</p> <p>This process will have to both download (read) and upload (write) the entire snapshot(s) due to the different encryption keys used in the source and destination repository. This <em>may incur higher bandwidth usage and costs</em> than expected during normal backup runs.</p> </div> <div class="admonition important"> <p class="admonition-title">Important</p> <p>The copying process does not re-chunk files, which may break deduplication between the files copied and files already stored in the destination repository. This means that copied files, which existed in both the source and destination repository, <em>may occupy up to twice their space</em> in the destination repository. See below for how to avoid this.</p> </div> <p>The destination repository is specified with <code class="docutils literal notranslate"><span class="pre">--repo2</span></code> or can be read from a file specified via <code class="docutils literal notranslate"><span class="pre">--repository-file2</span></code>. Both of these options can also set as environment variables <code class="docutils literal notranslate"><span class="pre">$RESTIC_REPOSITORY2</span></code> or <code class="docutils literal notranslate"><span class="pre">$RESTIC_REPOSITORY_FILE2</span></code> respectively. For the destination repository the password can be read from a file <code class="docutils literal notranslate"><span class="pre">--password-file2</span></code> or from a command <code class="docutils literal notranslate"><span class="pre">--password-command2</span></code>. Alternatively the environment variables <code class="docutils literal notranslate"><span class="pre">$RESTIC_PASSWORD_COMMAND2</span></code> and <code class="docutils literal notranslate"><span class="pre">$RESTIC_PASSWORD_FILE2</span></code> can be used. It is also possible to directly pass the password via <code class="docutils literal notranslate"><span class="pre">$RESTIC_PASSWORD2</span></code>. The key which should be used for decryption can be selected by passing its ID via the flag <code class="docutils literal notranslate"><span class="pre">--key-hint2</span></code> or the environment variable <code class="docutils literal notranslate"><span class="pre">$RESTIC_KEY_HINT2</span></code>.</p> <div class="admonition note"> <p class="admonition-title">Note</p> <p>In case the source and destination repository use the same backend, the configuration options and environment variables used to configure the backend may apply to both repositories – for example it might not be possible to specify different accounts for the source and destination repository. You can avoid this limitation by using the rclone backend along with remotes which are configured in rclone.</p> </div> <section id="filtering-snapshots-to-copy"> <h3>Filtering snapshots to copy<a class="headerlink" href="#filtering-snapshots-to-copy" title="Permalink to this headline">¶</a></h3> <p>The list of snapshots to copy can be filtered by host, path in the backup and / or a comma-separated tag list:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo copy --repo2 /srv/restic-repo-copy --host luigi --path /srv --tag foo,bar </pre></div> </div> <p>It is also possible to explicitly specify the list of snapshots to copy, in which case only these instead of all snapshots will be copied:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo copy --repo2 /srv/restic-repo-copy 410b18a2 4e5d5487 latest </pre></div> </div> </section> <section id="ensuring-deduplication-for-copied-snapshots"> <h3>Ensuring deduplication for copied snapshots<a class="headerlink" href="#ensuring-deduplication-for-copied-snapshots" title="Permalink to this headline">¶</a></h3> <p>Even though the copy command can transfer snapshots between arbitrary repositories, deduplication between snapshots from the source and destination repository may not work. To ensure proper deduplication, both repositories have to use the same parameters for splitting large files into smaller chunks, which requires additional setup steps. With the same parameters restic will for both repositories split identical files into identical chunks and therefore deduplication also works for snapshots copied between these repositories.</p> <p>The chunker parameters are generated once when creating a new (destination) repository. That is for a copy destination repository we have to instruct restic to initialize it using the same chunker parameters as the source repository:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo-copy init --repo2 /srv/restic-repo --copy-chunker-params </pre></div> </div> <p>Note that it is not possible to change the chunker parameters of an existing repository.</p> </section> </section> <section id="checking-integrity-and-consistency"> <h2>Checking integrity and consistency<a class="headerlink" href="#checking-integrity-and-consistency" title="Permalink to this headline">¶</a></h2> <p>Imagine your repository is saved on a server that has a faulty hard drive, or even worse, attackers get privileged access and modify the files in your repository with the intention to make you restore malicious data:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">echo</span> <span class="s2">"boom"</span> > /srv/restic-repo/index/de30f3231ca2e6a59af4aa84216dfe2ef7339c549dc11b09b84000997b139628 </pre></div> </div> <p>Trying to restore a snapshot which has been modified as shown above will yield an error:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo --no-cache restore c23e491f --target /tmp/restore-work <span class="go">...</span> <span class="go">Fatal: unable to load index de30f323: load <index/de30f3231c>: invalid data returned</span> </pre></div> </div> <p>In order to detect these things before they become a problem, it’s a good idea to regularly use the <code class="docutils literal notranslate"><span class="pre">check</span></code> command to test whether your repository is healthy and consistent, and that your precious backup data is unharmed. There are two types of checks that can be performed:</p> <ul class="simple"> <li><p>Structural consistency and integrity, e.g. snapshots, trees and pack files (default)</p></li> <li><p>Integrity of the actual data that you backed up (enabled with flags, see below)</p></li> </ul> <p>To verify the structure of the repository, issue the <code class="docutils literal notranslate"><span class="pre">check</span></code> command. If the repository is damaged like in the example above, <code class="docutils literal notranslate"><span class="pre">check</span></code> will detect this and yield the same error as when you tried to restore:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo check <span class="go">...</span> <span class="go">load indexes</span> <span class="go">error: error loading index de30f323: load <index/de30f3231c>: invalid data returned</span> <span class="go">Fatal: LoadIndex returned errors</span> </pre></div> </div> <p>If the repository structure is intact, restic will show that no errors were found:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /src/restic-repo check <span class="go">...</span> <span class="go">load indexes</span> <span class="go">check all packs</span> <span class="go">check snapshots, trees and blobs</span> <span class="go">no errors were found</span> </pre></div> </div> <p>By default, the <code class="docutils literal notranslate"><span class="pre">check</span></code> command does not verify that the actual pack files on disk in the repository are unmodified, because doing so requires reading a copy of every pack file in the repository. To tell restic to also verify the integrity of the pack files in the repository, use the <code class="docutils literal notranslate"><span class="pre">--read-data</span></code> flag:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo check --read-data <span class="go">...</span> <span class="go">load indexes</span> <span class="go">check all packs</span> <span class="go">check snapshots, trees and blobs</span> <span class="go">read all data</span> <span class="gp">[0:00] 100.00% </span><span class="m">3</span> / <span class="m">3</span> items <span class="go">duration: 0:00</span> <span class="go">no errors were found</span> </pre></div> </div> <div class="admonition note"> <p class="admonition-title">Note</p> <p>Since <code class="docutils literal notranslate"><span class="pre">--read-data</span></code> has to download all pack files in the repository, beware that it might incur higher bandwidth costs than usual and also that it takes more time than the default <code class="docutils literal notranslate"><span class="pre">check</span></code>.</p> </div> <p>Alternatively, use the <code class="docutils literal notranslate"><span class="pre">--read-data-subset</span></code> parameter to check only a subset of the repository pack files at a time. It supports two ways to select a subset. One selects a specific range of pack files, the other selects a random percentage of pack files.</p> <p>Use <code class="docutils literal notranslate"><span class="pre">--read-data-subset=n/t</span></code> to check only a subset of the repository pack files at a time. The parameter takes two values, <code class="docutils literal notranslate"><span class="pre">n</span></code> and <code class="docutils literal notranslate"><span class="pre">t</span></code>. When the check command runs, all pack files in the repository are logically divided in <code class="docutils literal notranslate"><span class="pre">t</span></code> (roughly equal) groups, and only files that belong to group number <code class="docutils literal notranslate"><span class="pre">n</span></code> are checked. For example, the following commands check all repository pack files over 5 separate invocations:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo check --read-data-subset<span class="o">=</span><span class="m">1</span>/5 <span class="gp">$ </span>restic -r /srv/restic-repo check --read-data-subset<span class="o">=</span><span class="m">2</span>/5 <span class="gp">$ </span>restic -r /srv/restic-repo check --read-data-subset<span class="o">=</span><span class="m">3</span>/5 <span class="gp">$ </span>restic -r /srv/restic-repo check --read-data-subset<span class="o">=</span><span class="m">4</span>/5 <span class="gp">$ </span>restic -r /srv/restic-repo check --read-data-subset<span class="o">=</span><span class="m">5</span>/5 </pre></div> </div> <p>Use <code class="docutils literal notranslate"><span class="pre">--read-data-subset=n%</span></code> to check a randomly choosen subset of the repository pack files. It takes one parameter, <code class="docutils literal notranslate"><span class="pre">n</span></code>, the percentage of pack files to check as an integer or floating point number. This will not guarantee to cover all available pack files after sufficient runs, but it is easy to automate checking a small subset of data after each backup. For a floating point value the following command may be used:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo check --read-data-subset<span class="o">=</span><span class="m">2</span>.5% </pre></div> </div> <p>When checking bigger subsets you most likely specify the percentage as an integer:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r /srv/restic-repo check --read-data-subset<span class="o">=</span><span class="m">10</span>% </pre></div> </div> </section> </section> </div> </div> <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer"> <a href="040_backup.html" class="btn btn-neutral float-left" title="Backing up" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a> <a href="050_restore.html" class="btn btn-neutral float-right" title="Restoring from backup" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a> </div> <hr/> <div role="contentinfo"> <p>© Copyright 2024, restic authors.</p> </div> Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. </footer> </div> </div> </section> </div> <script> jQuery(function () { SphinxRtdTheme.Navigation.enable(true); }); </script> </body> </html>

| ver. 1.4 | Github | . | PHP 8.2.28 | Generation time: 0.02 | proxy | phpinfo | Settings