pub-solar/obs-portal
5
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

Lots of documentation for new development setup as well as upgrade guide

Browse source
This commit is contained in:
Paul Bienkowski 2021-11-13 02:46:15 +01:00
parent da33d814ae
commit c353d2afc1
2 changed files with 105 additions and 124 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

198
README.md
View file

@ -25,48 +25,85 @@ git submodule update --init --recursive
## Production setup ## Production setup
**Note:** There is now a guide for a dockerized deployment in the There is a guide for a deployment based on docker in the
[deployment](deployment) folder. [deployment](deployment) folder. Lots of non-docker deployment strategy are
possible, but they are not "officially" supported, so please do not expect the
authors of the software to assist in troubleshooting.
You should be familiar with managing a Linux server. If not, find a suitable This is a rather complex application, and it is expected that you know the
guide first. This will only give a rough outline of the steps to take, you must basics of deploying a modern web application securely onto a production server.
make sure to properly set up and secure your server yourself. We are sorry that we cannot guide you through all the details of that, as we
just don't have the capacities to do so. Please research the respective topics
first. If you struggle with application-specific issues, please let us know, we
might be able to assist with those.
Please note that you will always need to install your own reverse proxy that
terminates TLS for you and handles certificates. We do not support TLS directly
in the application, instead, please use this prefered method.
### Migrating (Production)
Migrations are not implemented yet. Once we need them, we'll add them and
document the usage here.
### Upgrading from v0.2 to v0.3
After v0.2 we switched the underlying technology of the API and the database.
We now have no more MongoDB, instead, everything has moved to the PostgreSQL
installation. For development setups, it is advised to just reset the whole
state (remove the `local` folder) and start fresh. For production upgrades,
please follow the relevant section in [`UPGRADING.md`](./UPGRADING.md).
1. Create a user for running the application. It is not recommended to run as a
user that is also used for other things. Do not run as root!
2. Clone the repository.
3. Install `node` of at least version 15, and also `npm`. You can use the
[Node Version Manager (nvm)](https://github.com/nvm-sh/nvm), since this project
contains a `.nvmrc`. Just run `nvm install` followed by `nvm use`.
4. Run `npm ci` in the `api` and `frontend` directories to install dependencies.
5. Install and configure a MongoDB somewhere.
6. Copy `api/config.json.example` to `api/config.json` and change it to suit
your setup. Make sure to only use https URLs. Generate secure secrets.
Customize client IDs.
7. Copy `frontend/src/config.json.example` to `frontend/src/config.json` and
adjust.
8. Run `npm run build` in the frontend directory. This needs to be done after
changing the config, so if you did something wrong, re-run the build. If you
run your frontend on a non-root URL, run the build with the
`PUBLIC_URL=/prefix` environment variable.
9. Create a systemd-user service, or use tmux, or do whatever you like to start
the API service (`npm start` in `api/` directory). It should run as the
dedicated user, and expose `127.0.0.1:3000`. Use `PORT` environment variable
if you want to change the port. Make sure the service starts with
environment `NODE_ENV=production`, too.
10. Configure nginx or your reverse-proxy of choice to forward API requests to
the API port, and serve static files from the `frontend/build/` folder.
## Development setup ## Development setup
We've moved the whole development setup into Docker to make it easy for We've moved the whole development setup into Docker to make it easy for
everyone to get involved. After sucessfully [installing Docker everyone to get involved.
Engine](https://docs.docker.com/engine/install/) as well as [Docker
Compose](https://docs.docker.com/compose/install/) onto your machine, and ### Install docker
cloning the repository, all you need to do is:
Please [install Docker Engine](https://docs.docker.com/engine/install/) as well as
[Docker Compose](https://docs.docker.com/compose/install/) onto your machine.
Then clone the repository as described above.
### Configure Keycloak
Login will not be possible until you configure the keycloak realm correctly. Boot your keycloak instance:
```bash ```bash
docker-compose up -d --build frontend docker-compose up -d --build keycloak
```
Now navigate to http://localhost:3003/ and follow these steps:
* Click "Administration Console" and log in with `admin` / `admin`
* Hover over the realm name on the top left and click "Add realm"
* Name the Realm `OBS Dev` (spelling matters) and create it
* In the sidebar, navigate to Configure -> Clients, and click "Create" on the top right
* Client ID is `portal`. Hit "Save".
* In the Tab "Settings", edit the new client's "Access Type" to `confidential`
and enter as "Valid Redirect URIs": `http://localhost:3000/login/redirect`,
then "Save"
* Under "Credentials", copy the "Secret" and paste it into `api/config.dev.py`
as `KEYCLOAK_CLIENT_SECRET`. Please do not commit this change to git.
* In the sidebar, navigate to Manage -> Users, and click "Add user" on the top right.
* Give the user a name (e.g. `test`), leave the rest as-is.
* Under the tab "Credentials", set a new password, and make it non-temporary.
Press "Set Password".
We are going to automate this process. For now, you will have to repeat it
every time you reset the keycloak datbaase, which is inside the PostgreSQL. The
script `api/tools/reset_database.py` does *not* affect the state of the
keycloak database, however, so this should be rather rare.
### Boot the application
Now you can run the remaining parts of the application:
```bash
docker-compose up -d --build api worker frontend
``` ```
If this does not work, please open an issue and describe the problem you're If this does not work, please open an issue and describe the problem you're
@ -77,97 +114,10 @@ http://localhost:3000 -- but you probably only need to access the frontend for
testing. The frontend dev server also proxies all unknown requests to the API, testing. The frontend dev server also proxies all unknown requests to the API,
so the frontend always just requests data at its own URL. so the frontend always just requests data at its own URL.
## Running without docker ### Migrating (Development)
If you don't like docker, or want to run this in production without it, you can Migrations are not implemented yet. Once we need them, we'll add them and
do so as well. Our Docker setup is simply a slight wrapper around very simple document the usage here.
JavaScript packages that you can install yourself as usual, with `npm install`.
The API can be started with `npm start` inside its folder. The frontend
development server uses `npm start` as well, while building a production
version of the frontend happens with `npm run build`.
To connect the parts together, please have a look at what we're doing in the
"official" setup of docker, i.e. in `docker-compose.yaml`, the `Dockerfile`s
and in the respective `package.json` of the service. If you've done this kind
of thing before, it's not that hard. Otherwise, ask on Slack and there will be
somebody to help you ;)
## Running in production
You are advised not to use the dockerized mongodb service and instead do a
proper MongoDB setup on a server that is backed up and secured.
You can run the application in docker, but it is prefered to run it as a restricted
user in its own directory somewhere where it cannot escape ;)
The frontend should be built using `npm run build` and then served from a
proper web server, such as nginx or apache. See the instructions at
create-react-app concerning [deployment of an app](http://cra.link/deployment).
You are advised to virtualize your server for security reason, and separate
this whole application from other parts of your server system.
Also please install a reverse proxy that terminates TLS for you and handles
certificates. We do not support TLS directly in the application, instead,
please use this prefered method. This reverse proxy can also handle static file
serving for the frontend, no need for two separate server processes.
## Migrating
Sometimes your database will have to be migrated. The docker setup should do
this automatically, but if it does not work, you can run the following
commands:
```bash
# if running locally
(cd api/; npm run migrate:up)
# if running in docker
docker-compose run --rm api npm run migrate:up
````
## Custom MongoDB installation
If you have your own MongoDB instance running somewhere, you can set the
environment variable `MONGODB_URL` when starting the server, and it will read
that URL for connecting.
export MONGODB_URL=mongodb://user:password@mongodb.example.com/obs-app-database
This does not work when using docker-compose, in that case, you will have to
modify the `docker-compose.yaml` to include that URL.
## E-Mail Setup
By default in development mode mails are not sent, but instead the mail data is
logged to the console. This can be overriden with the `--devSendMails` flag if
you start the application like so: `npm run dev -- --devSendMails`.
Mails are also always sent in production mode!
For actually sending e-mails the mailserver, sender, user and password for the
SMTP server need to be specified as environment variables:
* `MAILUSER` -- the smtp mailbox login name
* `MAILPW` -- password for the mailbox
* `MAILSERVER` -- the hostname of the SMTP server, e.g. `mail.example.com`
* `MAILSENDER` -- sender name, e.g. `noreply@example.com`
Full command example:
```bash
MAILSERVER=mail.example.com MAILSENDER=noreply@example.com \
MAILUSER=my_mail_login MAILPW=hunter2 \
npm run dev -- --devSendMails
```
All of this of course is not too important if you're developing locally. To get
to the logged email content that *would* have been sent, check your docker log:
```bash
docker-compose logs -f api
```
## Tileserver generation ## Tileserver generation

31
UPGRADING.md Normal file
View file

@ -0,0 +1,31 @@
# Upgrading
This document describes the general steps to upgrade between major changes.
Simple migrations, e.g. for adding schema changes, are not documented
explicitly. Once we implement them, their usage will be described in the
[README](./README.md).
## v0.2 to v0.3 (MongoDB to PostgreSQL)
* Shut down all services
* Obviously, now is a good time to perform a full backup ;)
* Update the codebase
* Update docker-compose.yaml from the example. Leave the MongoDB service in
place for now, but update all other service descriptions. You can remove
`redis` already.
* Start up the `mongo` and `postgres` services
* Build the new images (e.g. with `docker-compose build`)
* Configure your API. The example config file is `api/config.py.example`, and
it will need to be mounted to `api/config.py` in the container. Ignore the
Keycloak options for now.
* Run the database reset script: `docker-compose run --rm api python tools/reset_database.py`
* Run the database migration script: `docker-compose run --rm api python
tools/import_from_mongodb.py mongodb://mongo/obs`. You might need to adjust
the URL.
* Shut down the `mongo` service, you can now remove it from docker-compose.yaml
* Start `keycloak` and configure it, similarly to how it was configured in the
development setup (but choose more secure options). Update the API config
file to match your keycloak configuration.
* Start `api`, `worker` and `frontend`