Lots of documentation for new development setup as well as upgrade guide
This commit is contained in:
Paul Bienkowski
parent
da33d814ae
commit
c353d2afc1
198
README.md
198
README.md
|
|
@ -25,48 +25,85 @@ git submodule update --init --recursive
|
|||
|
||||
## Production setup
|
||||
|
||||
**Note:** There is now a guide for a dockerized deployment in the
|
||||
[deployment](deployment) folder.
|
||||
There is a guide for a deployment based on docker in the
|
||||
[deployment](deployment) folder. Lots of non-docker deployment strategy are
|
||||
possible, but they are not "officially" supported, so please do not expect the
|
||||
authors of the software to assist in troubleshooting.
|
||||
|
||||
You should be familiar with managing a Linux server. If not, find a suitable
|
||||
guide first. This will only give a rough outline of the steps to take, you must
|
||||
make sure to properly set up and secure your server yourself.
|
||||
This is a rather complex application, and it is expected that you know the
|
||||
basics of deploying a modern web application securely onto a production server.
|
||||
We are sorry that we cannot guide you through all the details of that, as we
|
||||
just don't have the capacities to do so. Please research the respective topics
|
||||
first. If you struggle with application-specific issues, please let us know, we
|
||||
might be able to assist with those.
|
||||
|
||||
Please note that you will always need to install your own reverse proxy that
|
||||
terminates TLS for you and handles certificates. We do not support TLS directly
|
||||
in the application, instead, please use this prefered method.
|
||||
|
||||
### Migrating (Production)
|
||||
|
||||
Migrations are not implemented yet. Once we need them, we'll add them and
|
||||
document the usage here.
|
||||
|
||||
### Upgrading from v0.2 to v0.3
|
||||
|
||||
After v0.2 we switched the underlying technology of the API and the database.
|
||||
We now have no more MongoDB, instead, everything has moved to the PostgreSQL
|
||||
installation. For development setups, it is advised to just reset the whole
|
||||
state (remove the `local` folder) and start fresh. For production upgrades,
|
||||
please follow the relevant section in [`UPGRADING.md`](./UPGRADING.md).
|
||||
|
||||
1. Create a user for running the application. It is not recommended to run as a
|
||||
user that is also used for other things. Do not run as root!
|
||||
2. Clone the repository.
|
||||
3. Install `node` of at least version 15, and also `npm`. You can use the
|
||||
[Node Version Manager (nvm)](https://github.com/nvm-sh/nvm), since this project
|
||||
contains a `.nvmrc`. Just run `nvm install` followed by `nvm use`.
|
||||
4. Run `npm ci` in the `api` and `frontend` directories to install dependencies.
|
||||
5. Install and configure a MongoDB somewhere.
|
||||
6. Copy `api/config.json.example` to `api/config.json` and change it to suit
|
||||
your setup. Make sure to only use https URLs. Generate secure secrets.
|
||||
Customize client IDs.
|
||||
7. Copy `frontend/src/config.json.example` to `frontend/src/config.json` and
|
||||
adjust.
|
||||
8. Run `npm run build` in the frontend directory. This needs to be done after
|
||||
changing the config, so if you did something wrong, re-run the build. If you
|
||||
run your frontend on a non-root URL, run the build with the
|
||||
`PUBLIC_URL=/prefix` environment variable.
|
||||
9. Create a systemd-user service, or use tmux, or do whatever you like to start
|
||||
the API service (`npm start` in `api/` directory). It should run as the
|
||||
dedicated user, and expose `127.0.0.1:3000`. Use `PORT` environment variable
|
||||
if you want to change the port. Make sure the service starts with
|
||||
environment `NODE_ENV=production`, too.
|
||||
10. Configure nginx or your reverse-proxy of choice to forward API requests to
|
||||
the API port, and serve static files from the `frontend/build/` folder.
|
||||
|
||||
## Development setup
|
||||
|
||||
We've moved the whole development setup into Docker to make it easy for
|
||||
everyone to get involved. After sucessfully [installing Docker
|
||||
Engine](https://docs.docker.com/engine/install/) as well as [Docker
|
||||
Compose](https://docs.docker.com/compose/install/) onto your machine, and
|
||||
cloning the repository, all you need to do is:
|
||||
everyone to get involved.
|
||||
|
||||
### Install docker
|
||||
|
||||
Please [install Docker Engine](https://docs.docker.com/engine/install/) as well as
|
||||
[Docker Compose](https://docs.docker.com/compose/install/) onto your machine.
|
||||
|
||||
Then clone the repository as described above.
|
||||
|
||||
### Configure Keycloak
|
||||
|
||||
|
||||
Login will not be possible until you configure the keycloak realm correctly. Boot your keycloak instance:
|
||||
|
||||
```bash
|
||||
docker-compose up -d --build frontend
|
||||
docker-compose up -d --build keycloak
|
||||
```
|
||||
|
||||
Now navigate to http://localhost:3003/ and follow these steps:
|
||||
|
||||
* Click "Administration Console" and log in with `admin` / `admin`
|
||||
* Hover over the realm name on the top left and click "Add realm"
|
||||
* Name the Realm `OBS Dev` (spelling matters) and create it
|
||||
* In the sidebar, navigate to Configure -> Clients, and click "Create" on the top right
|
||||
* Client ID is `portal`. Hit "Save".
|
||||
* In the Tab "Settings", edit the new client's "Access Type" to `confidential`
|
||||
and enter as "Valid Redirect URIs": `http://localhost:3000/login/redirect`,
|
||||
then "Save"
|
||||
* Under "Credentials", copy the "Secret" and paste it into `api/config.dev.py`
|
||||
as `KEYCLOAK_CLIENT_SECRET`. Please do not commit this change to git.
|
||||
* In the sidebar, navigate to Manage -> Users, and click "Add user" on the top right.
|
||||
* Give the user a name (e.g. `test`), leave the rest as-is.
|
||||
* Under the tab "Credentials", set a new password, and make it non-temporary.
|
||||
Press "Set Password".
|
||||
|
||||
We are going to automate this process. For now, you will have to repeat it
|
||||
every time you reset the keycloak datbaase, which is inside the PostgreSQL. The
|
||||
script `api/tools/reset_database.py` does *not* affect the state of the
|
||||
keycloak database, however, so this should be rather rare.
|
||||
|
||||
### Boot the application
|
||||
|
||||
Now you can run the remaining parts of the application:
|
||||
|
||||
```bash
|
||||
docker-compose up -d --build api worker frontend
|
||||
```
|
||||
|
||||
If this does not work, please open an issue and describe the problem you're
|
||||
|
|
@ -77,97 +114,10 @@ http://localhost:3000 -- but you probably only need to access the frontend for
|
|||
testing. The frontend dev server also proxies all unknown requests to the API,
|
||||
so the frontend always just requests data at its own URL.
|
||||
|
||||
## Running without docker
|
||||
### Migrating (Development)
|
||||
|
||||
If you don't like docker, or want to run this in production without it, you can
|
||||
do so as well. Our Docker setup is simply a slight wrapper around very simple
|
||||
JavaScript packages that you can install yourself as usual, with `npm install`.
|
||||
The API can be started with `npm start` inside its folder. The frontend
|
||||
development server uses `npm start` as well, while building a production
|
||||
version of the frontend happens with `npm run build`.
|
||||
|
||||
To connect the parts together, please have a look at what we're doing in the
|
||||
"official" setup of docker, i.e. in `docker-compose.yaml`, the `Dockerfile`s
|
||||
and in the respective `package.json` of the service. If you've done this kind
|
||||
of thing before, it's not that hard. Otherwise, ask on Slack and there will be
|
||||
somebody to help you ;)
|
||||
|
||||
## Running in production
|
||||
|
||||
You are advised not to use the dockerized mongodb service and instead do a
|
||||
proper MongoDB setup on a server that is backed up and secured.
|
||||
|
||||
You can run the application in docker, but it is prefered to run it as a restricted
|
||||
user in its own directory somewhere where it cannot escape ;)
|
||||
|
||||
The frontend should be built using `npm run build` and then served from a
|
||||
proper web server, such as nginx or apache. See the instructions at
|
||||
create-react-app concerning [deployment of an app](http://cra.link/deployment).
|
||||
|
||||
You are advised to virtualize your server for security reason, and separate
|
||||
this whole application from other parts of your server system.
|
||||
|
||||
Also please install a reverse proxy that terminates TLS for you and handles
|
||||
certificates. We do not support TLS directly in the application, instead,
|
||||
please use this prefered method. This reverse proxy can also handle static file
|
||||
serving for the frontend, no need for two separate server processes.
|
||||
|
||||
## Migrating
|
||||
|
||||
Sometimes your database will have to be migrated. The docker setup should do
|
||||
this automatically, but if it does not work, you can run the following
|
||||
commands:
|
||||
|
||||
```bash
|
||||
# if running locally
|
||||
(cd api/; npm run migrate:up)
|
||||
|
||||
# if running in docker
|
||||
docker-compose run --rm api npm run migrate:up
|
||||
````
|
||||
|
||||
## Custom MongoDB installation
|
||||
|
||||
If you have your own MongoDB instance running somewhere, you can set the
|
||||
environment variable `MONGODB_URL` when starting the server, and it will read
|
||||
that URL for connecting.
|
||||
|
||||
export MONGODB_URL=mongodb://user:password@mongodb.example.com/obs-app-database
|
||||
|
||||
This does not work when using docker-compose, in that case, you will have to
|
||||
modify the `docker-compose.yaml` to include that URL.
|
||||
|
||||
|
||||
## E-Mail Setup
|
||||
|
||||
By default in development mode mails are not sent, but instead the mail data is
|
||||
logged to the console. This can be overriden with the `--devSendMails` flag if
|
||||
you start the application like so: `npm run dev -- --devSendMails`.
|
||||
|
||||
Mails are also always sent in production mode!
|
||||
|
||||
For actually sending e-mails the mailserver, sender, user and password for the
|
||||
SMTP server need to be specified as environment variables:
|
||||
|
||||
* `MAILUSER` -- the smtp mailbox login name
|
||||
* `MAILPW` -- password for the mailbox
|
||||
* `MAILSERVER` -- the hostname of the SMTP server, e.g. `mail.example.com`
|
||||
* `MAILSENDER` -- sender name, e.g. `noreply@example.com`
|
||||
|
||||
Full command example:
|
||||
|
||||
```bash
|
||||
MAILSERVER=mail.example.com MAILSENDER=noreply@example.com \
|
||||
MAILUSER=my_mail_login MAILPW=hunter2 \
|
||||
npm run dev -- --devSendMails
|
||||
```
|
||||
|
||||
All of this of course is not too important if you're developing locally. To get
|
||||
to the logged email content that *would* have been sent, check your docker log:
|
||||
|
||||
```bash
|
||||
docker-compose logs -f api
|
||||
```
|
||||
Migrations are not implemented yet. Once we need them, we'll add them and
|
||||
document the usage here.
|
||||
|
||||
## Tileserver generation
|
||||
|
||||
|
|
|
|||
31
UPGRADING.md
Normal file
31
UPGRADING.md
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
# Upgrading
|
||||
|
||||
This document describes the general steps to upgrade between major changes.
|
||||
Simple migrations, e.g. for adding schema changes, are not documented
|
||||
explicitly. Once we implement them, their usage will be described in the
|
||||
[README](./README.md).
|
||||
|
||||
|
||||
## v0.2 to v0.3 (MongoDB to PostgreSQL)
|
||||
|
||||
* Shut down all services
|
||||
* Obviously, now is a good time to perform a full backup ;)
|
||||
* Update the codebase
|
||||
* Update docker-compose.yaml from the example. Leave the MongoDB service in
|
||||
place for now, but update all other service descriptions. You can remove
|
||||
`redis` already.
|
||||
* Start up the `mongo` and `postgres` services
|
||||
* Build the new images (e.g. with `docker-compose build`)
|
||||
* Configure your API. The example config file is `api/config.py.example`, and
|
||||
it will need to be mounted to `api/config.py` in the container. Ignore the
|
||||
Keycloak options for now.
|
||||
* Run the database reset script: `docker-compose run --rm api python tools/reset_database.py`
|
||||
* Run the database migration script: `docker-compose run --rm api python
|
||||
tools/import_from_mongodb.py mongodb://mongo/obs`. You might need to adjust
|
||||
the URL.
|
||||
* Shut down the `mongo` service, you can now remove it from docker-compose.yaml
|
||||
* Start `keycloak` and configure it, similarly to how it was configured in the
|
||||
development setup (but choose more secure options). Update the API config
|
||||
file to match your keycloak configuration.
|
||||
* Start `api`, `worker` and `frontend`
|
||||
|
||||
Loading…
Reference in a new issue