How it works
Placeholders
The commands and policies on this page use the placeholders below. Replace each placeholder before you run a command.
Values that start with
CODE_STORAGE_ come from us. The policies that we send you have them filled
in. You do not need to change them.
Before you start
You need:- An AWS account and the AWS CLI version 2, signed in with permission to create buckets and set bucket policies.
jq,zstd,git, andgit-lfsto verify and restore the export.
1. Create the bucket
Create a new bucket for the export. Set Object Ownership to Bucket owner enforced, so that your account owns every object that we upload. The export checks this setting and stops before it writes anything if the setting is different.In
us-east-1, omit --create-bucket-configuration. S3 rejects that option in this region.Encrypt with your own KMS key (optional)
By default, the export encrypts objects with SSE-S3. To use SSE-KMS, create a customer-managed key in the same region as the bucket:2. Send us the destination
Email support@pierre.co with:- The destination folder as an S3 URI:
s3://YOUR_BUCKET/YOUR_FOLDER/ - The region of the bucket:
YOUR_REGION - For SSE-KMS only, the key ARN:
YOUR_KMS_KEY_ARN
. or ..
segments, empty segments such as a//b, or the characters *, ?, \, and %.
We reply with a bucket policy, and with a key policy statement if you use SSE-KMS. Both have every
value filled in for your organization and folder.
3. Apply the policies
Bucket policy
The bucket policy that we send has this form:YOUR_BUCKET and YOUR_FOLDER in the policy match your destination. The conditions
limit access to the export job for your organization. A job for another organization cannot use this
policy, even though it runs under the same role.
Save the policy as code-storage-policy.json. If the bucket has no policy, apply it as it is:
Key policy statement (SSE-KMS only)
For SSE-KMS, we also send a statement for the key policy. It lets the export job encrypt objects with your key and read them back to verify them:code-storage-key-statement.json and add it to the existing key policy. Keep the
existing statements, so that your account can still use the key and decrypt the export.
4. Stop using your repositories during the export
Agree on a start time with us. Before that time, stop all reads and changes to your repositories: clones, fetches, pushes, reads and commits through the API, imports, syncs, and repository creation or deletion. Let any operation in progress finish. Keep everything stopped until the export is complete. A read can move a repository out of cold storage while the export archives it, which can make the export fail. The export is not an atomic snapshot across repositories. A change during the export can invalidate a repository’s archive. If the set of repositories changes, the export stops, and we start a new export in a new folder. Do not change or delete files in the export folder while the export runs.5. Track progress
status.json in the export folder shows progress. It updates after each batch of repositories.
repositories.json is the completion marker. It appears only after every repository has a
verified archive and LFS copy. Until it exists, the export is not complete.
6. Download and verify
Download the export folder and check every file against its recorded SHA-256 checksum:repositories matches the number of repositories you expect
and the script prints no mismatch or failure. git fsck can print dangling lines, such as for a
deleted tag, and notice: No default references for an empty repository. These lines are not
errors.
7. Restore a repository
Each line incatalog/*.jsonl describes one repository:
url is the repository’s name in Code Storage. Use it to match each id to a repository. Each
archive is a bare Git repository with all refs and history. The previous step extracted each one
into restored/REPO_ID.git.
To restore a repository, create an empty repository on your new Git host and push to it:
git push --mirror does not set the default branch on the new host. Set it to the catalog’s
default_branch in your Git host’s repository settings, or with its API. For example, on GitHub:
restored/REPO_ID.git/config
before you use the repository elsewhere.
Restore LFS objects
Iflfs_objects is more than 0, copy the LFS objects into the restored repository and push them
to the new host. Review any .lfsconfig in the repository first, because it can name an old LFS
URL.
RESTORE.md in the export folder has the same steps.
8. Remove our access
After you verify the export, remove our statements from the bucket policy:PierreExportUseKey statement from the key policy:
What the export contains
The export contains Git repositories and LFS objects. It does not contain organization settings,
access keys, webhooks, or other configuration.