/ README.md
README.md
1 Journalist 2 ---------- 3 [](https://github.com/mrusme/journalist/actions/workflows/tests.yml) 4 [](https://github.com/mrusme/journalist/releases) 5 [](https://hub.docker.com/r/mrusme/journalist) 6 7  8 9 [](https://matrix.to/#/%21PHlbgZTdrhjkCJrfVY%3Amatrix.org) 11 12 Journalist. An RSS aggregator. 13 14 15 ## What is `journalist`? 16 17 Journalist is an RSS aggregator that can sync subscriptions and read/unread 18 items across multiple clients without requiring a special client-side 19 integration. Clients can use Journalist by simply subscribing to its 20 personalized RSS feed. 21 22 Journalist aims to become a self-hosted alternative to services like Feedly, 23 Feedbin and others. It aims to offer a similar set of features like FreshRSS, 24 NewsBlur and Miniflux while being easier to set up/maintain and overall more 25 lightweight. 26 27 Find out more about Journalist [here](https://xn--gckvb8fzb.com/journalist-v1/). 28 If you're looking for pre-v1.0.0 versions of Journalist, please check out the 29 [v0 branch](https://github.com/mrusme/journalist/tree/v0). *v1.0.0 and later 30 versions are not compatible to / upgradeable from pre-v1.0.0 versions!* 31 32 33 ## Usage 34 35 Journalist is a single binary service can be run on any Linux/Unix machine 36 by setting the required configuration values and launching the `journalist` 37 program. 38 39 Before using Journalist from an RSS client, it first requires 40 [configuration](#configuration) and [deployment](#deployment). 41 42 ### Getting Started 43 44 As soon as Journalist was [configured](#configuration) and 45 [deployed](#deployment) properly, a new user can be added using the admin user 46 that Journalist creates automatically (default login: `admin`:`admin`). 47 48 First, make sure to export `JOURNALIST_API_URL` in the current terminal session: 49 50 ```sh 51 $ export JOURNALIST_API_URL="http://127.0.0.1:8000/api/v1" 52 ``` 53 54 Then, using [Redacteur](#redacteur), a new user can be added like this: 55 56 ```sh 57 $ JOURNALIST_API_USERNAME=admin JOURNALIST_API_PASSWORD=admin \ 58 ./redacteur add user 59 Username: johndoe 60 Password: MySecretPassword123 61 Role (admin/[user]): user 62 ``` 63 64 Next, a new QAT (*Quick Access Token*) for the user can be issued: 65 66 ```sh 67 $ JOURNALIST_API_USERNAME=johndoe JOURNALIST_API_PASSWORD=MySecretPassword123 \ 68 ./redacteur add token 69 Token name: FeederAndroidClient 70 ``` 71 72 Copy the `token` from the JSON response, as this is required to subscribe to the 73 Journalist feed. 74 75 Next, add a new feed to the user (a.k.a. *subscribe to*): 76 77 ```sh 78 $ JOURNALIST_API_USERNAME=johndoe JOURNALIST_API_PASSWORD=MySecretPassword123 \ 79 ./redacteur add feed 80 URL: https://xn--gckvb8fzb.com 81 Name: マリウス 82 Group: Journals 83 ``` 84 85 Feel free to add further feeds the same way. `Group` describes a *folder* 86 underneath the feed should be grouped. Groups can be named freely. 87 88 With the *Quick Access Token* (`token`) that was copied previously, the 89 following URL can now be added to any RSS feed reader: 90 91 ``` 92 http://127.0.0.1:8000/web/subscriptions?qat=TOKEN-HERE 93 ``` 94 95 More information and RSS feed URLs can be found under [Web](#web). 96 97 98 ## Configuration 99 100 Journalist will read its config either from a file or from environment 101 variables. Every configuration key available in the 102 example [`journalist.toml`](examples/etc/journalist.toml) can be exported as 103 environment variable, by separating scopes using `_` and prepend `JOURNALIST` to 104 it. For example, the following configuration: 105 106 ```toml 107 [Server] 108 BindIP = "0.0.0.0" 109 ``` 110 111 ... can also be specified as an environment variable: 112 113 ```sh 114 export JOURNALIST_SERVER_BINDIP="0.0.0.0" 115 ``` 116 117 Journalist will try to read the `journalist.toml` file from one of the following 118 paths: 119 120 - `/etc/journalist.toml` 121 - `$XDG_CONFIG_HOME/journalist.toml` 122 - `$HOME/.config/journalist.toml` 123 - `$HOME/journalist.toml` 124 - `$PWD/journalist.toml` 125 126 127 ### Database 128 129 Journalist requires a database to store users and subscriptions. Supported 130 database types are SQLite, PostgreSQL and MySQL. The database can be configured 131 using the `JOURNALIST_DATABASE_TYPE` and `JOURNALIST_DATABASE_CONNECTION` env, 132 or the `Database.Type` and `Database.Connection` config properties. 133 134 **WARNING:** If you do not specify a database configuration, Journalist will use 135 an in-memory SQLite database! As soon as Journalist shuts down, all data 136 inside the in-memory database is gone! 137 138 139 #### SQLite File Example 140 141 ```toml 142 [Database] 143 Type = "sqlite3" 144 Connection = "file:my-database.sqlite?cache=shared&_fk=1" 145 ``` 146 147 148 #### PostgreSQL Example *(using Docker for PostgreSQL)* 149 150 Run the database: 151 152 ```sh 153 docker run -it --name postgres \ 154 -e POSTGRES_PASSWORD=postgres \ 155 -e POSTGRES_DB=journalist \ 156 -p 127.0.0.1:5432:5432 \ 157 -d postgres:alpine 158 ``` 159 160 Configure `Database.Type` and `Database.Connection`: 161 162 ```toml 163 [Database] 164 Type = "postgres" 165 Connection = "host=127.0.0.1 port=5432 dbname=journalist user=postgres password=postgres" 166 ``` 167 168 169 #### MySQL Example 170 171 ```toml 172 [Database] 173 Type = "mysql" 174 Connection = "mysqluser:mysqlpassword@tcp(mysqlhost:port)/database?parseTime=true" 175 ``` 176 177 178 ### Deployment 179 180 #### Custom 181 182 All that's needed is a [configuration](#configuration) and Journalist can be 183 launched by e.g. running `./journalist` in a terminal. 184 185 186 #### Supervisor 187 188 To run Journalist via `supervisord`, create a config like this inside 189 `/etc/supervisord.conf` or `/etc/supervisor/conf.d/journalist.conf`: 190 191 ```ini 192 [program:journalist] 193 command=/path/to/binary/of/journalist 194 process_name=%(program_name)s 195 numprocs=1 196 directory=/home/journalist 197 autostart=true 198 autorestart=unexpected 199 startsecs=10 200 startretries=3 201 exitcodes=0 202 stopsignal=TERM 203 stopwaitsecs=10 204 user=journalist 205 redirect_stderr=false 206 stdout_logfile=/var/log/journalist.out.log 207 stdout_logfile_maxbytes=1MB 208 stdout_logfile_backups=10 209 stdout_capture_maxbytes=1MB 210 stdout_events_enabled=false 211 stderr_logfile=/var/log/journalist.err.log 212 stderr_logfile_maxbytes=1MB 213 stderr_logfile_backups=10 214 stderr_capture_maxbytes=1MB 215 stderr_events_enabled=false 216 ``` 217 218 **Note:** It is advisable to run Journalist under its own, dedicated daemon 219 user (`journalist` in this example), so make sure to either adjust `directory` 220 as well as `user` or create a user called `journalist`. 221 222 223 #### OpenBSD rc 224 225 As before, create a configuration file under `/etc/journalist.toml`. 226 227 Then copy the [example rc.d script](examples/etc/rc.d/journalist) to 228 `/etc/rc.d/journalist` and copy the binary to e.g. 229 `/usr/local/bin/journalist`. Last but not least, update the `/etc/rc.conf.local` 230 file to contain the following line: 231 232 ```conf 233 journalist_user="_journalist" 234 ``` 235 236 It is advisable to run journalist as a dedicated user, hence create the 237 `_journalist` daemon account or adjust the line above according to your setup. 238 239 You can now run Journalist by enabling and starting the service: 240 241 ```sh 242 rcctl enable journalist 243 rcctl start journalist 244 ``` 245 246 247 #### systemd 248 249 TODO 250 251 252 #### Docker 253 254 Official images are available on Docker Hub at 255 [mrusme/journalist](https://hub.docker.com/r/mrusme/journalist) 256 and can be pulled using the following command: 257 258 ```sh 259 docker pull mrusme/journalist 260 ``` 261 262 GitHub release versions are available as Docker image tags (e.g. `1.0.0`). 263 The `latest` image tag contains the latest code of the `master` branch. 264 265 It's possible to build journalist locally as a Docker container like this: 266 267 ```sh 268 docker build -t journalist:latest . 269 ``` 270 271 It can then be run using the following command: 272 273 ```sh 274 docker run -it --rm --name journalist \ 275 -e JOURNALIST_... \ 276 -e JOURNALIST_... \ 277 -p 0.0.0.0:8000:8000 \ 278 journalist:latest 279 ``` 280 281 Alternatively a configuration TOML can be passed into the container like so: 282 283 ```sh 284 docker run -it --rm --name journalist \ 285 -v /path/to/my/local/journalist.toml:/etc/journalist.toml \ 286 -p 0.0.0.0:8000:8000 \ 287 journalist:latest 288 ``` 289 290 291 #### Kubernetes 292 293 TODO 294 295 296 #### Render 297 298 Fork this repo into your GitHub account, adjust the 299 [`render.yaml`](render.yaml) accordingly and connect the forked repo [on 300 Render](https://dashboard.render.com/select-repo?type=blueprint). 301 302 Alternatively, you can also directly connect this public repo. 303 304 305 #### Heroku 306 307 [](https://heroku.com/deploy?template=https://github.com/mrusme/journalist) 308 309 310 #### DigitalOcean App Platform 311 312 [](https://cloud.digitalocean.com/apps/new?repo=https://github.com/mrusme/journalist/tree/master&refcode=9d48825ddae1) 313 314 Alternatively, fork this repo into your GitHub account, adjust the 315 [`.do/app.yaml`](.do/app.yaml) accordingly and connect the forked repo [on 316 DigitalOcean](https://cloud.digitalocean.com/apps/new). 317 318 319 #### DigitalOcean Function 320 321 Available soon. 322 323 324 #### Aamazon Web Services Lambda Function 325 326 TODO 327 328 329 #### Google Cloud Function 330 331 ```sh 332 gcloud functions deploy GCFHandler --runtime go116 --trigger-http 333 ``` 334 335 TODO: Database 336 337 338 ## API 339 340 Journalist provides an HTTP REST API for managing user accounts, tokens and 341 feeds, which is available through the `/api/v1` endpoint. A full OpenAPI/Swagger 342 documentation of the API can be found inside the [`docs/`](docs/) folder. 343 344 ### Redacteur 345 346 This repository comes with a handy client for the Journalist API called 347 [*Redacteur*](redacteur). Redacteur can be used to perform actions on the API, 348 either by manually specifying the exact API request (`redacteur perform ...`) or 349 by using a shorthand function like `create user`, which runs interactively. 350 351 Find out more by running `redacteur help`. 352 353 354 ## Web 355 356 `/web` is the HTTP web endpoint of Journalist that serves aggregated RSS feeds 357 as well as *action* endpoints that allow for example marking items as read. 358 359 To subscribe to a Journalist user's aggregated RSS feed a *Quick Access Token* 360 is required. It can be generated using [Redacteur](#redacteur). 361 362 With the `QAT`, any RSS feed reader can subscribe to the following URL: 363 364 ``` 365 <JOURNALIST_SERVER_ENDPOINT_WEB>/subscriptions?qat=<TOKEN> 366 ``` 367 368 Additionally, subscriptions can be separated by *group*, simply by adding the 369 `group` parameter to the URL: 370 371 ``` 372 <JOURNALIST_SERVER_ENDPOINT_WEB>/subscriptions?qat=<TOKEN>&group=Journals 373 ``` 374 375 With that, only feeds within the *Journal* group will be included in the RSS 376 feed. 377 378 ### Mark as Read 379 380 Feed items can be marked as read using the inline Journalist menu that is 381 injected on the top of every RSS item. It contains a link to an *actions 382 endpoint* of Journalist that will mark either a single item or a specific range 383 of items as read. This will result in these items not showing up in the 384 Journalist subscription feed anymore. This way every other client that will 385 eventually refresh the feed won't *see* these items anymore and hopefully not 386 display them. 387 388 You might need to adjust client settings in order to disable caching of items. 389 Additionally, if a client has previously synced the items, it might not 390 automatically remove them from the feed. Whether and how good this works depends 391 on the client's implementation. 392 393 394 ## Development 395 396 First, install all required dependencies by running the following command in the 397 repository folder: 398 399 ```sh 400 make install-deps 401 ``` 402 403 You can then build Journalist by running `make`: 404 405 ```sh 406 make 407 ``` 408 409 This will build a binary called `journalist`. 410 411