File manager

File manager - Edit - /usr/share/doc/restic/html/030_preparing_a_new_repo.html

Back
<!DOCTYPE html> <html class="writer-html5" lang="en" > <head> <meta charset="utf-8" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Preparing a new repository — restic 0.12.1 documentation</title> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="_static/css/restic.css" type="text/css" /> <link rel="shortcut icon" href="_static/favicon.ico"/> <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script> <script src="_static/jquery.js"></script> <script src="_static/underscore.js"></script> <script src="_static/doctools.js"></script> <script src="_static/js/theme.js"></script> <link rel="index" title="Index" href="genindex.html" /> <link rel="search" title="Search" href="search.html" /> <link rel="next" title="Backing up" href="040_backup.html" /> <link rel="prev" title="Installation" href="020_installation.html" /> </head> <body class="wy-body-for-nav"> <div class="wy-grid-for-nav"> <nav data-toggle="wy-nav-shift" class="wy-nav-side"> <div class="wy-side-scroll"> <div class="wy-side-nav-search" > <a href="index.html" class="icon icon-home"> restic <img src="_static/logo.png" class="logo" alt="Logo"/> </a> <div class="version"> 0.12.1 </div> <div role="search"> <form id="rtd-search-form" class="wy-form" action="search.html" method="get"> <input type="text" name="q" placeholder="Search docs" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu"> <ul class="current"> <li class="toctree-l1"><a class="reference internal" href="010_introduction.html">Introduction</a></li> <li class="toctree-l1"><a class="reference internal" href="020_installation.html">Installation</a></li> <li class="toctree-l1 current"><a class="current reference internal" href="#">Preparing a new repository</a><ul> <li class="toctree-l2"><a class="reference internal" href="#local">Local</a></li> <li class="toctree-l2"><a class="reference internal" href="#sftp">SFTP</a></li> <li class="toctree-l2"><a class="reference internal" href="#rest-server">REST Server</a></li> <li class="toctree-l2"><a class="reference internal" href="#amazon-s3">Amazon S3</a></li> <li class="toctree-l2"><a class="reference internal" href="#minio-server">Minio Server</a></li> <li class="toctree-l2"><a class="reference internal" href="#wasabi">Wasabi</a></li> <li class="toctree-l2"><a class="reference internal" href="#alibaba-cloud-aliyun-object-storage-system-oss">Alibaba Cloud (Aliyun) Object Storage System (OSS)</a></li> <li class="toctree-l2"><a class="reference internal" href="#openstack-swift">OpenStack Swift</a></li> <li class="toctree-l2"><a class="reference internal" href="#backblaze-b2">Backblaze B2</a></li> <li class="toctree-l2"><a class="reference internal" href="#microsoft-azure-blob-storage">Microsoft Azure Blob Storage</a></li> <li class="toctree-l2"><a class="reference internal" href="#google-cloud-storage">Google Cloud Storage</a></li> <li class="toctree-l2"><a class="reference internal" href="#other-services-via-rclone">Other Services via rclone</a></li> <li class="toctree-l2"><a class="reference internal" href="#password-prompt-on-windows">Password prompt on Windows</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="040_backup.html">Backing up</a></li> <li class="toctree-l1"><a class="reference internal" href="045_working_with_repos.html">Working with repositories</a></li> <li class="toctree-l1"><a class="reference internal" href="050_restore.html">Restoring from backup</a></li> <li class="toctree-l1"><a class="reference internal" href="060_forget.html">Removing backup snapshots</a></li> <li class="toctree-l1"><a class="reference internal" href="070_encryption.html">Encryption</a></li> <li class="toctree-l1"><a class="reference internal" href="075_scripting.html">Scripting</a></li> <li class="toctree-l1"><a class="reference internal" href="080_examples.html">Examples</a></li> <li class="toctree-l1"><a class="reference internal" href="090_participating.html">Participating</a></li> <li class="toctree-l1"><a class="reference internal" href="100_references.html">References</a></li> <li class="toctree-l1"><a class="reference internal" href="110_talks.html">Talks</a></li> <li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li> <li class="toctree-l1"><a class="reference internal" href="manual_rest.html">Manual</a></li> <li class="toctree-l1"><a class="reference internal" href="developer_information.html">Developer Information</a></li> </ul> </div> </div> </nav> <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" > <i data-toggle="wy-nav-top" class="fa fa-bars"></i> <a href="index.html">restic</a> </nav> <div class="wy-nav-content"> <div class="rst-content"> <div role="navigation" aria-label="Page navigation"> <ul class="wy-breadcrumbs"> <li><a href="index.html" class="icon icon-home"></a> »</li> <li>Preparing a new repository</li> <li class="wy-breadcrumbs-aside"> <a href="_sources/030_preparing_a_new_repo.rst.txt" rel="nofollow"> View page source</a> </li> </ul> <hr/> </div> <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> <div itemprop="articleBody"> <section id="preparing-a-new-repository"> <h1>Preparing a new repository<a class="headerlink" href="#preparing-a-new-repository" title="Permalink to this headline">¶</a></h1> <p>The place where your backups will be saved is called a “repository”. This chapter explains how to create (“init”) such a repository. The repository can be stored locally, or on some remote server or service. We’ll first cover using a local repository; the remaining sections of this chapter cover all the other options. You can skip to the next chapter once you’ve read the relevant section here.</p> <p>For automated backups, restic accepts the repository location in the environment variable <code class="docutils literal notranslate"><span class="pre">RESTIC_REPOSITORY</span></code>. Restic can also read the repository location from a file specified via the <code class="docutils literal notranslate"><span class="pre">--repository-file</span></code> option or the environment variable <code class="docutils literal notranslate"><span class="pre">RESTIC_REPOSITORY_FILE</span></code>. For the password, several options exist:</p> <blockquote> <div><ul class="simple"> <li><p>Setting the environment variable <code class="docutils literal notranslate"><span class="pre">RESTIC_PASSWORD</span></code></p></li> <li><p>Specifying the path to a file with the password via the option <code class="docutils literal notranslate"><span class="pre">--password-file</span></code> or the environment variable <code class="docutils literal notranslate"><span class="pre">RESTIC_PASSWORD_FILE</span></code></p></li> <li><p>Configuring a program to be called when the password is needed via the option <code class="docutils literal notranslate"><span class="pre">--password-command</span></code> or the environment variable <code class="docutils literal notranslate"><span class="pre">RESTIC_PASSWORD_COMMAND</span></code></p></li> </ul> </div></blockquote> <section id="local"> <h2>Local<a class="headerlink" href="#local" title="Permalink to this headline">¶</a></h2> <p>In order to create a repository at <code class="docutils literal notranslate"><span class="pre">/srv/restic-repo</span></code>, run the following command and enter the same password twice:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic init --repo /srv/restic-repo <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository 085b3c76b9 at /srv/restic-repo</span> <span class="go">Please note that knowledge of your password is required to access the repository.</span> <span class="go">Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> <div class="admonition warning"> <p class="admonition-title">Warning</p> <p>Remembering your password is important! If you lose it, you won’t be able to access data stored in the repository.</p> </div> <div class="admonition warning"> <p class="admonition-title">Warning</p> <p>On Linux, storing the backup repository on a CIFS (SMB) share is not recommended due to compatibility issues. Either use another backend or set the environment variable <cite>GODEBUG</cite> to <cite>asyncpreemptoff=1</cite>. Refer to GitHub issue <a class="reference external" href="https://github.com/restic/restic/issues/2659">#2659</a> for further explanations.</p> </div> </section> <section id="sftp"> <h2>SFTP<a class="headerlink" href="#sftp" title="Permalink to this headline">¶</a></h2> <p>In order to backup data via SFTP, you must first set up a server with SSH and let it know your public key. Passwordless login is really important since restic fails to connect to the repository if the server prompts for credentials.</p> <p>Once the server is configured, the setup of the SFTP repository can simply be achieved by changing the URL scheme in the <code class="docutils literal notranslate"><span class="pre">init</span></code> command:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r sftp:user@host:/srv/restic-repo init <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository f1c6108821 at sftp:user@host:/srv/restic-repo</span> <span class="go">Please note that knowledge of your password is required to access the repository.</span> <span class="go">Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> <p>You can also specify a relative (read: no slash (<code class="docutils literal notranslate"><span class="pre">/</span></code>) character at the beginning) directory, in this case the dir is relative to the remote user’s home directory.</p> <p>Also, if the SFTP server is enforcing domain-confined users, you can specify the user this way: <code class="docutils literal notranslate"><span class="pre">user@domain@host</span></code>.</p> <div class="admonition note"> <p class="admonition-title">Note</p> <p>Please be aware that sftp servers do not expand the tilde character (<code class="docutils literal notranslate"><span class="pre">~</span></code>) normally used as an alias for a user’s home directory. If you want to specify a path relative to the user’s home directory, pass a relative path to the sftp backend.</p> </div> <p>If you need to specify a port number or IPv6 address, you’ll need to use URL syntax. E.g., the repository <code class="docutils literal notranslate"><span class="pre">/srv/restic-repo</span></code> on <code class="docutils literal notranslate"><span class="pre">[::1]</span></code> (localhost) at port 2222 with username <code class="docutils literal notranslate"><span class="pre">user</span></code> can be specified as</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sftp</span><span class="p">:</span><span class="o">//</span><span class="n">user</span><span class="o">@</span><span class="p">[::</span><span class="mi">1</span><span class="p">]:</span><span class="mi">2222</span><span class="o">//</span><span class="n">srv</span><span class="o">/</span><span class="n">restic</span><span class="o">-</span><span class="n">repo</span> </pre></div> </div> <p>Note the double slash: the first slash separates the connection settings from the path, while the second is the start of the path. To specify a relative path, use one slash.</p> <p>Alternatively, you can create an entry in the <code class="docutils literal notranslate"><span class="pre">ssh</span></code> configuration file, usually located in your home directory at <code class="docutils literal notranslate"><span class="pre">~/.ssh/config</span></code> or in <code class="docutils literal notranslate"><span class="pre">/etc/ssh/ssh_config</span></code>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Host</span> <span class="n">foo</span> <span class="n">User</span> <span class="n">bar</span> <span class="n">Port</span> <span class="mi">2222</span> </pre></div> </div> <p>Then use the specified host name <code class="docutils literal notranslate"><span class="pre">foo</span></code> normally (you don’t need to specify the user name in this case):</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ restic -r sftp:foo:/srv/restic-repo init </pre></div> </div> <p>You can also add an entry with a special host name which does not exist, just for use with restic, and use the <code class="docutils literal notranslate"><span class="pre">Hostname</span></code> option to set the real host name:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Host</span> <span class="n">restic</span><span class="o">-</span><span class="n">backup</span><span class="o">-</span><span class="n">host</span> <span class="n">Hostname</span> <span class="n">foo</span> <span class="n">User</span> <span class="n">bar</span> <span class="n">Port</span> <span class="mi">2222</span> </pre></div> </div> <p>Then use it in the backend specification:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ restic -r sftp:restic-backup-host:/srv/restic-repo init </pre></div> </div> <p>Last, if you’d like to use an entirely different program to create the SFTP connection, you can specify the command to be run with the option <code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">sftp.command="foobar"</span></code>.</p> <div class="admonition note"> <p class="admonition-title">Note</p> <p>Please be aware that sftp servers close connections when no data is received by the client. This can happen when restic is processing huge amounts of unchanged data. To avoid this issue add the following lines to the client’s .ssh/config file:</p> </div> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ServerAliveInterval</span> <span class="mi">60</span> <span class="n">ServerAliveCountMax</span> <span class="mi">240</span> </pre></div> </div> </section> <section id="rest-server"> <h2>REST Server<a class="headerlink" href="#rest-server" title="Permalink to this headline">¶</a></h2> <p>In order to backup data to the remote server via HTTP or HTTPS protocol, you must first set up a remote <a class="reference external" href="https://github.com/restic/rest-server">REST server</a> instance. Once the server is configured, accessing it is achieved by changing the URL scheme like this:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r rest:http://host:8000/ </pre></div> </div> <p>Depending on your REST server setup, you can use HTTPS protocol, password protection, multiple repositories or any combination of those features. The TCP/IP port is also configurable. Here are some more examples:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r rest:https://host:8000/ <span class="gp">$ </span>restic -r rest:https://user:pass@host:8000/ <span class="gp">$ </span>restic -r rest:https://user:pass@host:8000/my_backup_repo/ </pre></div> </div> <p>If you use TLS, restic will use the system’s CA certificates to verify the server certificate. When the verification fails, restic refuses to proceed and exits with an error. If you have your own self-signed certificate, or a custom CA certificate should be used for verification, you can pass restic the certificate filename via the <code class="docutils literal notranslate"><span class="pre">--cacert</span></code> option. It will then verify that the server’s certificate is contained in the file passed to this option, or signed by a CA certificate in the file. In this case, the system CA certificates are not considered at all.</p> <p>REST server uses exactly the same directory structure as local backend, so you should be able to access it both locally and via HTTP, even simultaneously.</p> </section> <section id="amazon-s3"> <h2>Amazon S3<a class="headerlink" href="#amazon-s3" title="Permalink to this headline">¶</a></h2> <p>Restic can backup data to any Amazon S3 bucket. However, in this case, changing the URL scheme is not enough since Amazon uses special security credentials to sign HTTP requests. By consequence, you must first setup the following environment variables with the credentials you obtained while creating the bucket.</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_ACCESS_KEY_ID</span><span class="o">=</span><MY_ACCESS_KEY> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_SECRET_ACCESS_KEY</span><span class="o">=</span><MY_SECRET_ACCESS_KEY> </pre></div> </div> <p>You can then easily initialize a repository that uses your Amazon S3 as a backend. If the bucket does not exist it will be created in the default location:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r s3:s3.amazonaws.com/bucket_name init <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository eefee03bbd at s3:s3.amazonaws.com/bucket_name</span> <span class="go">Please note that knowledge of your password is required to access the repository.</span> <span class="go">Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> <p>If needed, you can manually specify the region to use by either setting the environment variable <code class="docutils literal notranslate"><span class="pre">AWS_DEFAULT_REGION</span></code> or calling restic with an option parameter like <code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">s3.region="us-east-1"</span></code>. If the region is not specified, the default region is used. Afterwards, the S3 server (at least for AWS, <code class="docutils literal notranslate"><span class="pre">s3.amazonaws.com</span></code>) will redirect restic to the correct endpoint.</p> <p>Until version 0.8.0, restic used a default prefix of <code class="docutils literal notranslate"><span class="pre">restic</span></code>, so the files in the bucket were placed in a directory named <code class="docutils literal notranslate"><span class="pre">restic</span></code>. If you want to access a repository created with an older version of restic, specify the path after the bucket name like this:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r s3:s3.amazonaws.com/bucket_name/restic <span class="o">[</span>...<span class="o">]</span> </pre></div> </div> <p>For an S3-compatible server that is not Amazon (like Minio, see below), or is only available via HTTP, you can specify the URL to the server like this: <code class="docutils literal notranslate"><span class="pre">s3:http://server:port/bucket_name</span></code>.</p> <div class="admonition note"> <p class="admonition-title">Note</p> <p>restic expects <a class="reference external" href="https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro">path-style URLs</a> like for example <code class="docutils literal notranslate"><span class="pre">s3.us-west-2.amazonaws.com/bucket_name</span></code>. Virtual-hosted–style URLs like <code class="docutils literal notranslate"><span class="pre">bucket_name.s3.us-west-2.amazonaws.com</span></code>, where the bucket name is part of the hostname are not supported. These must be converted to path-style URLs instead, for example <code class="docutils literal notranslate"><span class="pre">s3.us-west-2.amazonaws.com/bucket_name</span></code>.</p> </div> <div class="admonition note"> <p class="admonition-title">Note</p> <p>Certain S3-compatible servers do not properly implement the <code class="docutils literal notranslate"><span class="pre">ListObjectsV2</span></code> API, most notably Ceph versions before v14.2.5. On these backends, as a temporary workaround, you can provide the <code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">s3.list-objects-v1=true</span></code> option to use the older <code class="docutils literal notranslate"><span class="pre">ListObjects</span></code> API instead. This option may be removed in future versions of restic.</p> </div> </section> <section id="minio-server"> <h2>Minio Server<a class="headerlink" href="#minio-server" title="Permalink to this headline">¶</a></h2> <p><a class="reference external" href="https://www.minio.io">Minio</a> is an Open Source Object Storage, written in Go and compatible with AWS S3 API.</p> <ul class="simple"> <li><p>Download and Install <a class="reference external" href="https://minio.io/downloads/#minio-server">Minio Server</a>.</p></li> <li><p>You can also refer to <a class="reference external" href="https://docs.minio.io">https://docs.minio.io</a> for step by step guidance on installation and getting started on Minio Client and Minio Server.</p></li> </ul> <p>You must first setup the following environment variables with the credentials of your Minio Server.</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_ACCESS_KEY_ID</span><span class="o">=</span><YOUR-MINIO-ACCESS-KEY-ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_SECRET_ACCESS_KEY</span><span class="o">=</span> <YOUR-MINIO-SECRET-ACCESS-KEY> </pre></div> </div> <p>Now you can easily initialize restic to use Minio server as a backend with this command.</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>./restic -r s3:http://localhost:9000/restic init <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository 6ad29560f5 at s3:http://localhost:9000/restic1</span> <span class="go">Please note that knowledge of your password is required to access</span> <span class="go">the repository. Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> </section> <section id="wasabi"> <h2>Wasabi<a class="headerlink" href="#wasabi" title="Permalink to this headline">¶</a></h2> <p><a class="reference external" href="https://wasabi.com">Wasabi</a> is a low cost AWS S3 conformant object storage provider. Due to it’s S3 conformance, Wasabi can be used as a storage provider for a restic repository.</p> <ul class="simple"> <li><p>Create a Wasabi bucket using the <a class="reference external" href="https://console.wasabisys.com">Wasabi Console</a>.</p></li> <li><p>Determine the correct Wasabi service URL for your bucket <a class="reference external" href="https://wasabi-support.zendesk.com/hc/en-us/articles/360015106031-What-are-the-service-URLs-for-Wasabi-s-different-regions-">here</a>.</p></li> </ul> <p>You must first setup the following environment variables with the credentials of your Wasabi account.</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_ACCESS_KEY_ID</span><span class="o">=</span><YOUR-WASABI-ACCESS-KEY-ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_SECRET_ACCESS_KEY</span><span class="o">=</span><YOUR-WASABI-SECRET-ACCESS-KEY> </pre></div> </div> <p>Now you can easily initialize restic to use Wasabi as a backend with this command.</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>./restic -r s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME> init <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository xxxxxxxxxx at s3:https://<WASABI-SERVICE-URL>/<WASABI-BUCKET-NAME></span> <span class="go">Please note that knowledge of your password is required to access</span> <span class="go">the repository. Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> </section> <section id="alibaba-cloud-aliyun-object-storage-system-oss"> <h2>Alibaba Cloud (Aliyun) Object Storage System (OSS)<a class="headerlink" href="#alibaba-cloud-aliyun-object-storage-system-oss" title="Permalink to this headline">¶</a></h2> <p><a class="reference external" href="https://www.alibabacloud.com/product/oss/">Alibaba OSS</a> is an encrypted, secure, cost-effective, and easy-to-use object storage service that enables you to store, back up, and archive large amounts of data in the cloud.</p> <p>Alibaba OSS is S3 compatible so it can be used as a storage provider for a restic repository with a couple of extra parameters.</p> <ul class="simple"> <li><p>Determine the correct <a class="reference external" href="https://www.alibabacloud.com/help/doc-detail/31837.htm">Alibaba OSS region endpoint</a> - this will be something like <code class="docutils literal notranslate"><span class="pre">oss-eu-west-1.aliyuncs.com</span></code></p></li> <li><p>You’ll need the region name too - this will be something like <code class="docutils literal notranslate"><span class="pre">oss-eu-west-1</span></code></p></li> </ul> <p>You must first setup the following environment variables with the credentials of your Alibaba OSS account.</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_ACCESS_KEY_ID</span><span class="o">=</span><YOUR-OSS-ACCESS-KEY-ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">AWS_SECRET_ACCESS_KEY</span><span class="o">=</span><YOUR-OSS-SECRET-ACCESS-KEY> </pre></div> </div> <p>Now you can easily initialize restic to use Alibaba OSS as a backend with this command.</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>./restic -o s3.bucket-lookup<span class="o">=</span>dns -o s3.region<span class="o">=</span><OSS-REGION> -r s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME> init <span class="go">enter password for new backend:</span> <span class="go">enter password again:</span> <span class="go">created restic backend xxxxxxxxxx at s3:https://<OSS-ENDPOINT>/<OSS-BUCKET-NAME></span> <span class="go">Please note that knowledge of your password is required to access</span> <span class="go">the repository. Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> <p>For example with an actual endpoint:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -o s3.bucket-lookup<span class="o">=</span>dns -o s3.region<span class="o">=</span>oss-eu-west-1 -r s3:https://oss-eu-west-1.aliyuncs.com/bucketname init </pre></div> </div> </section> <section id="openstack-swift"> <h2>OpenStack Swift<a class="headerlink" href="#openstack-swift" title="Permalink to this headline">¶</a></h2> <p>Restic can backup data to an OpenStack Swift container. Because Swift supports various authentication methods, credentials are passed through environment variables. In order to help integration with existing OpenStack installations, the naming convention of those variables follows the official Python Swift client:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp"># </span>For keystone v1 authentication <span class="gp">$ </span><span class="nb">export</span> <span class="nv">ST_AUTH</span><span class="o">=</span><MY_AUTH_URL> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">ST_USER</span><span class="o">=</span><MY_USER_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">ST_KEY</span><span class="o">=</span><MY_USER_PASSWORD> <span class="gp"># </span>For keystone v2 authentication <span class="o">(</span>some variables are optional<span class="o">)</span> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_AUTH_URL</span><span class="o">=</span><MY_AUTH_URL> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_REGION_NAME</span><span class="o">=</span><MY_REGION_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_USERNAME</span><span class="o">=</span><MY_USERNAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_PASSWORD</span><span class="o">=</span><MY_PASSWORD> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_TENANT_ID</span><span class="o">=</span><MY_TENANT_ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_TENANT_NAME</span><span class="o">=</span><MY_TENANT_NAME> <span class="gp"># </span>For keystone v3 authentication <span class="o">(</span>some variables are optional<span class="o">)</span> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_AUTH_URL</span><span class="o">=</span><MY_AUTH_URL> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_REGION_NAME</span><span class="o">=</span><MY_REGION_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_USERNAME</span><span class="o">=</span><MY_USERNAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_USER_ID</span><span class="o">=</span><MY_USER_ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_PASSWORD</span><span class="o">=</span><MY_PASSWORD> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_USER_DOMAIN_NAME</span><span class="o">=</span><MY_DOMAIN_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_USER_DOMAIN_ID</span><span class="o">=</span><MY_DOMAIN_ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_PROJECT_NAME</span><span class="o">=</span><MY_PROJECT_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_PROJECT_DOMAIN_NAME</span><span class="o">=</span><MY_PROJECT_DOMAIN_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_PROJECT_DOMAIN_ID</span><span class="o">=</span><MY_PROJECT_DOMAIN_ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_TRUST_ID</span><span class="o">=</span><MY_TRUST_ID> <span class="gp"># </span>For keystone v3 application credential authentication <span class="o">(</span>application credential id<span class="o">)</span> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_AUTH_URL</span><span class="o">=</span><MY_AUTH_URL> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_APPLICATION_CREDENTIAL_ID</span><span class="o">=</span><MY_APPLICATION_CREDENTIAL_ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_APPLICATION_CREDENTIAL_SECRET</span><span class="o">=</span><MY_APPLICATION_CREDENTIAL_SECRET> <span class="gp"># </span>For keystone v3 application credential authentication <span class="o">(</span>application credential name<span class="o">)</span> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_AUTH_URL</span><span class="o">=</span><MY_AUTH_URL> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_USERNAME</span><span class="o">=</span><MY_USERNAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_USER_DOMAIN_NAME</span><span class="o">=</span><MY_DOMAIN_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_APPLICATION_CREDENTIAL_NAME</span><span class="o">=</span><MY_APPLICATION_CREDENTIAL_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_APPLICATION_CREDENTIAL_SECRET</span><span class="o">=</span><MY_APPLICATION_CREDENTIAL_SECRET> <span class="gp"># </span>For authentication based on tokens <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_STORAGE_URL</span><span class="o">=</span><MY_STORAGE_URL> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">OS_AUTH_TOKEN</span><span class="o">=</span><MY_AUTH_TOKEN> </pre></div> </div> <p>Restic should be compatible with an <a class="reference external" href="https://docs.openstack.org/user-guide/common/cli-set-environment-variables-using-openstack-rc.html">OpenStack RC file</a> in most cases.</p> <p>Once environment variables are set up, a new repository can be created. The name of the Swift container and optional path can be specified. If the container does not exist, it will be created automatically:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r swift:container_name:/path init <span class="c1"># path is optional</span> <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository eefee03bbd at swift:container_name:/path</span> <span class="go">Please note that knowledge of your password is required to access the repository.</span> <span class="go">Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> <p>The policy of the new container created by restic can be changed using environment variable:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">SWIFT_DEFAULT_CONTAINER_POLICY</span><span class="o">=</span><MY_CONTAINER_POLICY> </pre></div> </div> </section> <section id="backblaze-b2"> <h2>Backblaze B2<a class="headerlink" href="#backblaze-b2" title="Permalink to this headline">¶</a></h2> <p>Restic can backup data to any Backblaze B2 bucket. You need to first setup the following environment variables with the credentials you can find in the dashboard on the “Buckets” page when signed into your B2 account:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">B2_ACCOUNT_ID</span><span class="o">=</span><MY_APPLICATION_KEY_ID> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">B2_ACCOUNT_KEY</span><span class="o">=</span><MY_APPLICATION_KEY> </pre></div> </div> <div class="admonition note"> <p class="admonition-title">Note</p> <p>As of version 0.9.2, restic supports both master and non-master <a class="reference external" href="https://www.backblaze.com/b2/docs/application_keys.html">application keys</a>. If using a non-master application key, ensure that it is created with at least <strong>read and write</strong> access to the B2 bucket. On earlier versions of restic, a master application key is required.</p> </div> <p>You can then initialize a repository stored at Backblaze B2. If the bucket does not exist yet and the credentials you passed to restic have the privilege to create buckets, it will be created automatically:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r b2:bucketname:path/to/repo init <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository eefee03bbd at b2:bucketname:path/to/repo</span> <span class="go">Please note that knowledge of your password is required to access the repository.</span> <span class="go">Losing your password means that your data is irrecoverably lost.</span> </pre></div> </div> <p>Note that the bucket name must be unique across all of B2.</p> <p>The number of concurrent connections to the B2 service can be set with the <code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">b2.connections=10</span></code> switch. By default, at most five parallel connections are established.</p> </section> <section id="microsoft-azure-blob-storage"> <h2>Microsoft Azure Blob Storage<a class="headerlink" href="#microsoft-azure-blob-storage" title="Permalink to this headline">¶</a></h2> <p>You can also store backups on Microsoft Azure Blob Storage. Export the Azure account name and key as follows:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">AZURE_ACCOUNT_NAME</span><span class="o">=</span><ACCOUNT_NAME> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">AZURE_ACCOUNT_KEY</span><span class="o">=</span><SECRET_KEY> </pre></div> </div> <p>Afterwards you can initialize a repository in a container called <code class="docutils literal notranslate"><span class="pre">foo</span></code> in the root path like this:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r azure:foo:/ init <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository a934bac191 at azure:foo:/</span> <span class="go">[...]</span> </pre></div> </div> <p>The number of concurrent connections to the Azure Blob Storage service can be set with the <code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">azure.connections=10</span></code> switch. By default, at most five parallel connections are established.</p> </section> <section id="google-cloud-storage"> <h2>Google Cloud Storage<a class="headerlink" href="#google-cloud-storage" title="Permalink to this headline">¶</a></h2> <p>Restic supports Google Cloud Storage as a backend and connects via a <a class="reference external" href="https://cloud.google.com/storage/docs/authentication#service_accounts">service account</a>.</p> <p>For normal restic operation, the service account must have the <code class="docutils literal notranslate"><span class="pre">storage.objects.{create,delete,get,list}</span></code> permissions for the bucket. These are included in the “Storage Object Admin” role. <code class="docutils literal notranslate"><span class="pre">restic</span> <span class="pre">init</span></code> can create the repository bucket. Doing so requires the <code class="docutils literal notranslate"><span class="pre">storage.buckets.create</span></code> permission (“Storage Admin” role). If the bucket already exists, that permission is unnecessary.</p> <p>To use the Google Cloud Storage backend, first <a class="reference external" href="https://cloud.google.com/storage/docs/authentication#generating-a-private-key">create a service account key</a> and download the JSON credentials file. Second, find the Google Project ID that you can see in the Google Cloud Platform console at the “Storage/Settings” menu. Export the path to the JSON key file and the project ID as follows:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">GOOGLE_PROJECT_ID</span><span class="o">=</span><span class="m">123123123123</span> <span class="gp">$ </span><span class="nb">export</span> <span class="nv">GOOGLE_APPLICATION_CREDENTIALS</span><span class="o">=</span><span class="nv">$HOME</span>/.config/gs-secret-restic-key.json </pre></div> </div> <p>Restic uses Google’s client library to generate <a class="reference external" href="https://developers.google.com/identity/protocols/application-default-credentials">default authentication material</a>, which means if you’re running in Google Container Engine or are otherwise located on an instance with default service accounts then these should work out of the box.</p> <p>Alternatively, you can specify an existing access token directly:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">GOOGLE_ACCESS_TOKEN</span><span class="o">=</span>ya29.a0AfH6SMC78... </pre></div> </div> <p>If <code class="docutils literal notranslate"><span class="pre">GOOGLE_ACCESS_TOKEN</span></code> is set all other authentication mechanisms are disabled. The access token must have at least the <code class="docutils literal notranslate"><span class="pre">https://www.googleapis.com/auth/devstorage.read_write</span></code> scope. Keep in mind that access tokens are short-lived (usually one hour), so they are not suitable if creating a backup takes longer than that, for instance.</p> <p>Once authenticated, you can use the <code class="docutils literal notranslate"><span class="pre">gs:</span></code> backend type to create a new repository in the bucket <code class="docutils literal notranslate"><span class="pre">foo</span></code> at the root path:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r gs:foo:/ init <span class="go">enter password for new repository:</span> <span class="go">enter password again:</span> <span class="go">created restic repository bde47d6254 at gs:foo2/</span> <span class="go">[...]</span> </pre></div> </div> <p>The number of concurrent connections to the GCS service can be set with the <code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">gs.connections=10</span></code> switch. By default, at most five parallel connections are established.</p> </section> <section id="other-services-via-rclone"> <h2>Other Services via rclone<a class="headerlink" href="#other-services-via-rclone" title="Permalink to this headline">¶</a></h2> <p>The program <a class="reference external" href="https://rclone.org/">rclone</a> can be used to access many other different services and store data there. First, you need to install and <a class="reference external" href="https://rclone.org/docs/">configure</a> rclone. The general backend specification format is <code class="docutils literal notranslate"><span class="pre">rclone:<remote>:<path></span></code>, the <code class="docutils literal notranslate"><span class="pre"><remote>:<path></span></code> component will be directly passed to rclone. When you configure a remote named <code class="docutils literal notranslate"><span class="pre">foo</span></code>, you can then call restic as follows to initiate a new repository in the path <code class="docutils literal notranslate"><span class="pre">bar</span></code> in the repo:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r rclone:foo:bar init </pre></div> </div> <p>Restic takes care of starting and stopping rclone.</p> <p>As a more concrete example, suppose you have configured a remote named <code class="docutils literal notranslate"><span class="pre">b2prod</span></code> for Backblaze B2 with rclone, with a bucket called <code class="docutils literal notranslate"><span class="pre">yggdrasil</span></code>. You can then use rclone to list files in the bucket like this:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>rclone ls b2prod:yggdrasil </pre></div> </div> <p>In order to create a new repository in the root directory of the bucket, call restic like this:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r rclone:b2prod:yggdrasil init </pre></div> </div> <p>If you want to use the path <code class="docutils literal notranslate"><span class="pre">foo/bar/baz</span></code> in the bucket instead, pass this to restic:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -r rclone:b2prod:yggdrasil/foo/bar/baz init </pre></div> </div> <p>Listing the files of an empty repository directly with rclone should return a listing similar to the following:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>rclone ls b2prod:yggdrasil/foo/bar/baz <span class="go"> 155 bar/baz/config</span> <span class="go"> 448 bar/baz/keys/4bf9c78049de689d73a56ed0546f83b8416795295cda12ec7fb9465af3900b44</span> </pre></div> </div> <p>Rclone can be <a class="reference external" href="https://rclone.org/docs/#environment-variables">configured with environment variables</a>, so for instance configuring a bandwidth limit for rclone can be achieved by setting the <code class="docutils literal notranslate"><span class="pre">RCLONE_BWLIMIT</span></code> environment variable:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span><span class="nb">export</span> <span class="nv">RCLONE_BWLIMIT</span><span class="o">=</span>1M </pre></div> </div> <p>For debugging rclone, you can set the environment variable <code class="docutils literal notranslate"><span class="pre">RCLONE_VERBOSE=2</span></code>.</p> <p>The rclone backend has two additional options:</p> <blockquote> <div><ul class="simple"> <li><p><code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">rclone.program</span></code> specifies the path to rclone, the default value is just <code class="docutils literal notranslate"><span class="pre">rclone</span></code></p></li> <li><p><code class="docutils literal notranslate"><span class="pre">-o</span> <span class="pre">rclone.args</span></code> allows setting the arguments passed to rclone, by default this is <code class="docutils literal notranslate"><span class="pre">serve</span> <span class="pre">restic</span> <span class="pre">--stdio</span> <span class="pre">--b2-hard-delete</span></code></p></li> </ul> </div></blockquote> <p>The reason for the <code class="docutils literal notranslate"><span class="pre">--b2-hard-delete</span></code> parameters can be found in the corresponding GitHub <a class="reference external" href="https://github.com/restic/restic/pull/1657#issuecomment-377707486">issue #1657</a>.</p> <p>In order to start rclone, restic will build a list of arguments by joining the following lists (in this order): <code class="docutils literal notranslate"><span class="pre">rclone.program</span></code>, <code class="docutils literal notranslate"><span class="pre">rclone.args</span></code> and as the last parameter the value that follows the <code class="docutils literal notranslate"><span class="pre">rclone:</span></code> prefix of the repository specification.</p> <p>So, calling restic like this</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -o rclone.program<span class="o">=</span><span class="s2">"/path/to/rclone"</span> <span class="se">\</span> -o rclone.args<span class="o">=</span><span class="s2">"serve restic --stdio --bwlimit 1M --b2-hard-delete --verbose"</span> <span class="se">\</span> -r rclone:b2:foo/bar </pre></div> </div> <p>runs rclone as follows:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>/path/to/rclone serve restic --stdio --bwlimit 1M --b2-hard-delete --verbose b2:foo/bar </pre></div> </div> <p>Manually setting <code class="docutils literal notranslate"><span class="pre">rclone.program</span></code> also allows running a remote instance of rclone e.g. via SSH on a server, for example:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -o rclone.program<span class="o">=</span><span class="s2">"ssh user@remotehost rclone"</span> -r rclone:b2:foo/bar </pre></div> </div> <p>With these options, restic works with local files. It uses rclone and credentials stored on <code class="docutils literal notranslate"><span class="pre">remotehost</span></code> to communicate with B2. All data (except credentials) is encrypted/decrypted locally, then sent/received via <code class="docutils literal notranslate"><span class="pre">remotehost</span></code> to/from B2.</p> <p>A more advanced version of this setup forbids specific hosts from removing files in a repository. See the <a class="reference external" href="https://ruderich.org/simon/notes/append-only-backups-with-restic-and-rclone">blog post by Simon Ruderich</a> for details.</p> <p>The rclone command may also be hard-coded in the SSH configuration or the user’s public key, in this case it may be sufficient to just start the SSH connection (and it’s irrelevant what’s passed after <code class="docutils literal notranslate"><span class="pre">rclone:</span></code> in the repository specification):</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>restic -o rclone.program<span class="o">=</span><span class="s2">"ssh user@host"</span> -r rclone:x </pre></div> </div> </section> <section id="password-prompt-on-windows"> <h2>Password prompt on Windows<a class="headerlink" href="#password-prompt-on-windows" title="Permalink to this headline">¶</a></h2> <p>At the moment, restic only supports the default Windows console interaction. If you use emulation environments like <a class="reference external" href="https://msys2.github.io/">MSYS2</a> or <a class="reference external" href="https://www.cygwin.com/">Cygwin</a>, which use terminals like <code class="docutils literal notranslate"><span class="pre">Mintty</span></code> or <code class="docutils literal notranslate"><span class="pre">rxvt</span></code>, you may get a password error.</p> <p>You can workaround this by using a special tool called <code class="docutils literal notranslate"><span class="pre">winpty</span></code> (look <a class="reference external" href="https://github.com/msys2/msys2/wiki/Porting">here</a> and <a class="reference external" href="https://github.com/rprichard/winpty">here</a> for detail information). On MSYS2, you can install <code class="docutils literal notranslate"><span class="pre">winpty</span></code> as follows:</p> <div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>pacman -S winpty <span class="gp">$ </span>winpty restic -r /srv/restic-repo init </pre></div> </div> </section> </section> </div> </div> <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer"> <a href="020_installation.html" class="btn btn-neutral float-left" title="Installation" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a> <a href="040_backup.html" class="btn btn-neutral float-right" title="Backing up" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a> </div> <hr/> <div role="contentinfo"> <p>© Copyright 2024, restic authors.</p> </div> Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. </footer> </div> </div> </section> </div> <script> jQuery(function () { SphinxRtdTheme.Navigation.enable(true); }); </script> </body> </html>

| ver. 1.4 | Github | . | PHP 8.2.28 | Generation time: 0.02 | proxy | phpinfo | Settings