# Getting Started

## Creating repositories

### First setup

If you're starting a new repository, go to that repository and then run
`copier copy --trust gl:znicholls/copier-notebook-based-book path/to/destination/repository`.
Copier will ask you the required questions and then setup your repository for you.

Alternatively, if you have the template repository cloned locally,
the path to the repository can be used instead of `gl:znicholls/copier-notebook-based-book`.

Copier will use the latest tagged release for generating a new project.
If you wish to use a specific commit/tag the `--vcs-ref` flag can be used
(`--vcs-ref HEAD` will use the most recent commit).

To run this step, you will need to have copier installed globally
(we can't make the virtual environment first as that is done with the `Makefile`,
which copier creates for us).

Once you have created your repository, there are then a number of further
steps which have to be done to get everything running as intended. The
sections below go through these steps.

#### GitLab

If you're hosting the repository on GitLab, do the steps in this section too.
Otherwise, skip this section.

In the GitLab page for your project, under Settings -> CI/CD, go to the
"Variables" heading.

Add the known GitLab host keys (these can be generated with e.g.
`ssh-keyscan gitlab.com`) to a variable called `SSH_KNOWN_HOSTS`. Make it of
file type and protected (so it can only be accessed in protected branches and
tags) but don't worry about masking it.

Now, create an ssh key without a passphrase. This can be done with e.g.
`ssh-keygen -f deploy_key -N ""`. You need to put the private key in a GitLab
variable called `SSH_PRIVATE_KEY`. Make it of type "File" so that it can
effectively be masked by GitLab. Make sure it ends with an empty line so that
it is properly parsed later on. Once again, make it a protected variable.

Next, add the public key as a deploy key to your project in Settings ->
Repository -> Deploy keys. Grant write permissions to the key and set an
expiry date if you want (after which time you'll need to rotate the keys).

Now, go to Settings -> Repository -> Protected tags and add "v\*" as a protected
tag. This will ensure that commits created with bump can also access the
variables we are setting up. By default, the main branch should be protected
so you shouldn't need to change the protected branches at all (but feel free
to check that main is actually protected and make it protected if main isn't
already protected for some reason).

If the repository has a protected main branch which disallows anyone from pushing,
the newly added deploy key must be added to the "Allowed to push and merge" user set.
If you don't do this you will get an error stating
"You are not allowed to push code to protected branches on this project.".
If that happens, contact your administrator as the CI may be in an odd state
that requires cleaning the existing docker containers.

If you wish, also add a
[pipeline schedule](https://docs.gitlab.com/ee/ci/pipelines/schedules.html#add-a-pipeline-schedule)
to run the scheduled jobs on a schedule of your choosing.
These jobs do automated maintenance checks,
like checking that the book builds from scratch.

Checklist:

- \[ \] `SSH_PRIVATE_KEY` CI variable containing **private** part of the SSH key
- \[ \] `SSH_KNOWN_HOSTS` CI variable containing the result of `ssh-keyscan gitlab.com`
- \[ \] Deploy key with write permissions using the **public** part of the SSH key
- \[ \] Protect the "v\*" git tag
- \[ \] Add a pipeline schedule for automated maintenance jobs

This section is based on the
[the commitizen docs](https://commitizen-tools.github.io/commitizen/tutorials/gitlab_ci/).

#### GitHub

If you're hosting the repository on GitHub, do the steps in this section too.
Otherwise, skip this section.

Create a [personal access token](https://docs.github.com/en/enterprise-server@3.4/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#about-personal-access-tokens)
that can be used to write to the repository as part of GitHub actions.
It is best to use the fine-grained tokens
and only give the token access to this repository.
The token will need "Contents" permissions,
specifically read and write access for "Contents".

If you can't create a token,
your organisation may need to enable personal access token access.
Please ask one of the lead developers for your organisation.

Once you have your token, add it to a repository secret
(Settings -> Secrets and variables -> Actions -> New repository secret)
called `PERSONAL_ACCESS_TOKEN`.

Checklist:

- \[ \] `PERSONAL_ACCESS_TOKEN` repository secrete containing a personal access token (for a single repository only) with read and write access for "Contents"

##### Pre-commit CI

On GitHub, we can make use of [pre-commit CI](https://pre-commit.ci/)
to run our pre-commit jobs (effectively our linting).
This is configured by our `.pre-commit-config.yaml` file.
The only step required to get it to run is to ensure
that pre-commit CI is configured so that it has access to this repository.
To do this, go to installations in your organisation's GitHub organisation page
(the URL will be something like https://github.com/organizations/my-org-name/settings/installations,
alternately you can get there by following
https://github.com/organizations/my-org-name -> Settings -> GitHub Apps)
then press 'configure' next to pre-commit CI.
Ensure that pre-commit CI has access to this repository or add access,
then you're all setup.
On each pull request, the pre-commit tasks will be run on all files
and any failures will be reported in the pull request page.

#### Read the Docs

At the moment, we only integrate with readthedocs.org
hence this setup only works for public repositories.
If you want to host docs for a private repository, you will need to get setup with
[Read the Docs for Business](https://about.readthedocs.com/about/).

If you haven't already, you will need to link your GitHub/GitLab account with RtD.
This can be done by (if you don't already have one)
creating an account with [RtD](https://readthedocs.org/)
and then going to
[Settings -> Connected services](https://readthedocs.org/accounts/social/connections/).
From here, the connection can be trivially added by pressing
"Connect to GitLab/GitHub" and following the prompts.

To add your project, go to your [RtD dashboard](https://readthedocs.org/dashboard/)
then press "Import a Project".
Pick your project from the list
then simply follow the setup instructions
(you can mostly just leave the defaults as they are,
the only thing we have done in the past is adding tags).

Once your project is working with RtD,
we recommend going to Admin, scrolling down/searching
and ticking the "Build pull requests for this project" box.
That will cause the docs to be built on all pull requests,
which gives fast feedback about whether a change will
a) break the docs and
b) cause the docs to be updated in the expected way.

It is also worth going to Versions and making sure that the "latest"
(generally the last commit merged to main)
and "stable" versions are both active
so that users can see docs for both these versions.

(updating-repositories)=

## Updating repositories

If you need to update your repository,
simply navigate to your repository and run `copier update`.
If you don't want to go through all the questions again
(the default answers are taken from last time you answered the questions),
use `copier update --force` instead.

By default, copier will use the most recent tag when updating the repository.
If you wish to use the current HEAD commit for your update,
run `copier update --vcs-ref=HEAD`.
This `--vcs-ref` option can also be used to specify a specific tag to apply.

When you update, there will likely be merge conflicts,
particularly in `pyproject.toml` related to versions.
If you use the `--conflict inline` option with `copier update`
then the diffs should be inline
(this is apparently experimental though, see
[here](https://copier.readthedocs.io/en/stable/updating/)).
These diffs can then be resolved manually.
The pre-commit config will make sure you don't miss conflicts
and accidentally commit merge conflict lines.

Any conflicts related to the `poetry.lock` file can be safety ignored
and a `poetry lock` should be run after updating to regenerate the lockfile.