Skip to content
Snippets Groups Projects
Commit 4532be85 authored by Welter, Felix's avatar Welter, Felix
Browse files

Rephrase; correct spelling; add deploy options

parent 45245138
No related branches found
No related tags found
No related merge requests found
......@@ -2,7 +2,7 @@
This repository acts as the entrypoint for the Related Items project.
It contains information for the overall project architecture
as well as links to the different parts and respositories
as well as links to the different parts and repositories
which are part of the Related Items project.
Authors: Fabian Rausch and Felix Welter
......@@ -10,7 +10,7 @@ Authors: Fabian Rausch and Felix Welter
# Quick links
- [bigbluebutton-html5 client](https://gitlab.rrz.uni-hamburg.de/wilps20/html5-modifications/-/tree/related-items/bigbluebutton-html5) (Use branch related-items)
- [bigbluebutton-html5](https://gitlab.rrz.uni-hamburg.de/wilps20/html5-modifications/-/tree/related-items/bigbluebutton-html5) (Use branch related-items)
- [kaldi-model-server](https://gitlab.rrz.uni-hamburg.de/wilps20/kaldi-model-server/-/tree/rtc-stream-docker) (Use branch rtc-stream-docker)
- [message-broker](https://gitlab.rrz.uni-hamburg.de/wilps20/message-broker)
- [relevant-terms-tagger](https://gitlab.rrz.uni-hamburg.de/wilps20/relevant-terms-tagger)
......@@ -21,32 +21,38 @@ Authors: Fabian Rausch and Felix Welter
# Context
This project was developed during the Master Project "Web Interfaces for Language Processing Systems" and was supervised
by Prof. Dr. Chris Biemann and Dr. Seid Muhie Yimam. In light of the Corona pandemic 2020 the project aims at improving the online conferencing software bigbluebutton. The Related Items project enables users to transcribe speech
by Prof. Dr. Chris Biemann and Dr. Seid Muhie Yimam. In light of the Corona pandemic 2020 the project aimed at improving the online conferencing software bigbluebutton. The Related Items project enables users to transcribe speech
and lookup related items (e.g. lecture slides or a wikipedia article) via an easy to use interface.
# Software architecture
The following image visualizes the software architecture.
![software architecture](architecture.png "software architecture")
Generally almost all services are dockerized and can be run on the same machine or on different servers each.
E.g. it works well to put all services on one machine except for the indices, which are hosted on different
The originally used setup has shown that it works well to put all services
on one machine except for the indices, which are hosted on different
servers for independent scaling.
We recommend putting each service behind an nginx proxy, which easily adds SSL support and improves performance.
We recommend putting each service behind an nginx proxy,
which easily adds SSL support and improves performance.
Especially support for SSL makes development more hasslefree.
## Standard request format
This is a standard format for request to and answers from an index (e.g. slide-index, wikipedia) or more
general sources of related items. This way, new sources for related items can easily be added.
This is a standard format for requests to and answers from an index (e.g. slide-index, wikipedia). This way, new sources for related items can easily be added.
The service just needs to adhere to the standard request format and a few lines are added to
the html5-client.
The micro service is queried via the POST endpoint: `/search`.
The request contains the POST params `term` and `context`.
The request contains the POST params `term`, `context` and `amount`.
Currently `amount` it usually set to three.
The micro service returns a json object containing the response `type`
and type-dependent data.
For image:
For images:
```
{
"type": "image",
......@@ -90,6 +96,7 @@ If no information was found:
An examples of an implementation with flask can be found in the slide-index repository.
## Docker
There are docker containers available for the following components:
- [modified kaldi-model-server](https://hub.docker.com/repository/docker/fwelter/wilps-kaldi-model-server)
- [slide-index](https://hub.docker.com/repository/docker/fwelter/wilps_slide_index)
......@@ -97,23 +104,24 @@ There are docker containers available for the following components:
- [wikipedia-search-service](https://hub.docker.com/r/fabianrausch/wikipedia-search-service)
- [relevant-terms-tagger](https://hub.docker.com/r/fabianrausch/relevant-terms-tagger)
Always use the latest version available.
The specific command to run each container can be found in the respective repositories.
Unless stated otherwise, always use the latest version available.
Specific instructions to run each container can be found in the respective repositories.
The html5-client is not dockerized since it adheres to the development guidelines of bigbluebutton.
The html5-client is not dockerized since it adheres to the [development guidelines of bigbluebutton](https://docs.bigbluebutton.org/2.2/dev.html#developing-the-html5-client).
The message-broker is a trivial nodejs script and can easily be run on a bigbluebutton server.
If you have no experience with docker please check out [docker installation](https://docs.docker.com/get-started/)
and [docker usage](https://docs.docker.com/get-started/part2/).
## Nginx
This section gives an overview of the nginx setup and a general installation procedure. Please note
that some steps may be redundant on your system and additional steps may be required (e.g. firewall exceptions)
that some steps may be redundant on your system and/or additional steps may be required (e.g. firewall exceptions)
dependent on your setup.
This setup also assumes that you have a registered (sub-)domain which points to your server(s),
since this is required for SSL certificates.
This guide also assumes that you have a registered (sub-)domain which points to your server(s),
since this is required for SSL certificates. If this requirement does not suite you, please
refer to the section *Alternatives to nginx+SSL setup* below
### Installation
```
......@@ -136,7 +144,7 @@ sudo apt-get install python-certbot-nginx
sudo certbot --nginx -d example.com
```
More details can be found here: https://www.digitalocean.com/community/tutorials/how-to-set-up-let-s-encrypt-with-nginx-server-blocks-on-ubuntu-16-04
More details regarding nginx and the SSL setup can be found here: https://www.digitalocean.com/community/tutorials/how-to-set-up-let-s-encrypt-with-nginx-server-blocks-on-ubuntu-16-04
### Nginx reverse proxy
Nginx needs to know where your service is running. This can be done with the following
......@@ -179,12 +187,34 @@ location <URL_PATH_PREFIX> {
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
}
```
*URL_PATH_PREFIX* This defines the matching path for the service. `/` will redirect all request to the service.
By using e.g. `/message_broker` and several location blocks, multiple services can run on one server.
*URL_PATH_PREFIX* This defines the matching path for the service. `/` will redirect all requests to the service.
By using e.g. `/msg_broker` and several location blocks, multiple services can run on one server.
*SERVICE_CONNECTION* This specifies where the service runs (e.g. http://127.0.0.1:8080/). Make sure, that
*SERVICE_CONNECTION* This specifies where the service runs (e.g. http://127.0.0.1:8080/). Make sure that
it has a trailing `/` so the *URL_PATH_PREFIX* is not send to the service.
The last two lines containing *Access-Control-Allow* are required for the service to be used from a different domain.
(e.g. the user participates in a conference at `bbb.uhh.de` but the service is hosted at `index.uhh.de`).
If you service runs on the same domain/server as the html5 client, these two lines are not required.
For more information, refer to [CORS](https://developer.mozilla.org/de/docs/Web/HTTP/CORS) and
[nginx CORS](https://enable-cors.org/server_nginx.html).
### Alternatives to nginx+SSL setup
These deployment options are untested for this project, however they are standard procedures
in other software development projects and could be more suitable if other individuals
want to setup the service system of the Related Items project.
**Option 1**
Host everything (including bigbluebutton) locally. [According to bigbluebutton this is
possible with an Ubuntu 16](https://docs.bigbluebutton.org/2.2/install.html#minimum-server-requirements)
machine. Please be aware that for BigBlueButton,
Kaldi and Elasticsearch a considerable amount of hardware ressources is needed.
16gb memory and a performant CPU (especially for Kaldi) should be available.
**Option 2**
Host the services on external servers, however instead of connecting regulary,
tunnel via SSH to each server and connect via localhost. In most browsers this
will prevent SSL warnings and errors.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment