<center>https://cdn.steemitimages.com/DQmZvBkuJePehh9pphd9JyH1VjvyoKv5Hjswd7o7JnwgRHF/Hivemind%20Live.jpg</center>

## Project Information
Repository: https://github.com/Jolly-Pirate/hivemind-docker
Project Name: hivemind-docker

## Introduction
Hive (https://github.com/steemit/hivemind) is a "consensus interpretation" layer for the Steem blockchain, maintaining the state of social features such as post feeds, follows, and communities. Written in Python, it synchronizes an SQL database with chain state, providing developers with a more flexible/extensible alternative to the raw steemd API.

I'm using docker-compose for this project because of its flexibility in managing multiple containers, and to minimize the use of long complicated docker cli commands. With dependency checks in the scripts, I covered many possibilities to make a hivemind deployment easy. The tricky and frustrating part was to get postgres to use a mapped local database outside of docker volumes. The reason for that is to have better control of the database and to simplify  redeployment to other servers, instead of having to redownload/reimport a dump which takes hours. Postgres is picky about permissions, plus its native command `initdb` was causing errors when synchronizing an imported hivemind dump. After solving the problems with some workarounds and thoroughly testing, hivemind-docker was finally ready.

---
### <center>An elephant and a whale can be friends</center>
<center>https://info.crunchydata.com/hubfs/DockerNPostgresLogos.png</center>

---

## Features
- High flexibility with simple commands
- Local storage of the database for easier redeployment in an infrastructure
- Cli commands to control the containers

## Requirements
- Docker Engine 18.06.0+
- PostgreSQL 10+
- 2.5GB of memory for hivemind synch process
- 250GB storage for the database

## Git Installation
```
git clone https://github.com/Jolly-Pirate/hivemind-docker.git
cd hivemind-docker
chmod +x run.sh
```

## Preparation
Do the following steps sequentially:

1. Create a .env file from the example and secure it\
`cp .env.example .env`
`chmod 700 .env`
2. Carefully edit all the variables in the .env file and save it\
`nano .env`
3. Preinstall tools and NTP synchronization\
`./run.sh preinstall`
4. Install docker and docker-compose\
`./run.sh installdocker`
5. Build the needed containers\
`./run.sh build`


## Running Postgres
Hivemind requires a postgres backend. Start postgres with these two commands consecutively:

`./run.sh initdb` (will initialize a fresh database cluster)\
`./run.sh start postgres` (will start postgres using the credentials from .env)

## Importing database dump
For an efficient way to get hivemind going with a short DB synchronization, download a database dump (from a daily postgres snapshot), and import it. The dump was done with PostgreSQL 10.8. ETA depends on your internet speed, storage and CPU.

NOTE: The download and import will run in screen sessions and automatically exit when complete. After the dump file is fully read, the import may seem stalled, but it's actually creating the indexes. Do not stop the process with `ctrl-c` or close the session . You can detach from the session with `ctrl-a-d`.

`./run.sh importdb`

**ETA ~3h**

## Running Hivemind
After the DB import, start hivemind to synchronize the missing blocks.

`./run.sh start hive`

**ETA ~1h**

## Running Hivemind and Postgres simultaneously
If you didn't do the 3 steps above, hivemind can be ran from scratch, the script will create a new postgres database, then synchronize it with an endpoint RPC server. However be advised that this approach is lenghthy, depending on your machine specs. For this *plug-n-play* solution, run the following command:

`./run.sh start all`

 **ETA few days**

To stop the hivemind and postgres containers

`./run.sh stop all`

If you already imported the database, doing `./run.sh start all` will resume synchronization from the last processed block.

## Testing Hivemind
Once hivemind is fully synchronized, you can test it by querying it on the port you defined in `.env`, for example `HIVEMIND_PORT=8080`:

`./run.sh testhive`

which runs this command:

`curl -s --data '[{"jsonrpc":"2.0", "method":"condenser_api.get_follow_count", "params":{"account":"initminer"}, "id":1}]' http://localhost:8080 | jq -r`

and gives this result:
```
[
  {
    "jsonrpc": "2.0",
    "result": {
      "account": "initminer",
      "following_count": 0,
      "follower_count": 15
    },
    "id": 1
  }
]
```

## Running Jussi
Jussi is an optional reverse proxy, its configuration won't be covered in this guide. You can check it out at https://github.com/steemit/jussi

Edit and place the DEV_config.json file in the hivemind-docker folder, then start it with:

`./run.sh start jussi`

You can test it with

`./run.sh testjussi`

## Checking the logs
At any time, you can check the logs with:

`./run.sh logs`

Press `ctrl-c` to stop following the logs.

## Project command options
The commands for managing the docker project are listed by typing:

`./run.sh`

Here's a summary of the available commands:
```
 preinstall    - preinstall tools and NTP synchronization
 installdocker - install docker and docker-compose
 build         - stop the running containers and (re)build all the images

 initdb        - initialize database cluster (e.g. postgresql database)
 importdb      - download and import the database dump

 start|stop|restart (e.g. start all)
           all - initdb+postgresql+hivemind
      postgres - postgresql container (with initdb dependency)
          hive - hivemind container (with postgresql dependency)
         jussi - jussi reverse proxy
 enter         - enter a container with bash shell; e.g. enter hive
 logs          - live logs of the running containers
 status        - check the containers status

 testhive      - test a hive API call to hivemind
 testjussi     - test a steemd API call to jussi

 dbsize        - check the database size
 dbactivity    - check the database activity
```

## Proof of work done
Github account: https://github.com/Jolly-Pirate

### Acknowledgment
Thanks to @emrebeyler. His guide on setting up hivemind was an inspiration for this project.
- https://steemit.com/hivemind/@emrebeyler/steps-to-restore-a-hivemind-database-snapshot
- https://steemit.com/hivemind/@emrebeyler/update-to-daily-hivemind-snapshots

Also, thanks for providing the community with a daily hivemind database snapshot.

<center><sub>Posted via https://www.palnet.io</sub>
[![](https://cdn.steemitimages.com/DQma2sXTD5C79ByLLUddrjzyztXK2xCbzz2xPo4FtZfVpxP/plcoin-onblack_gif.gif)](https://www.palnet.io)</center>

***
<center>
#### Available & Reliable. I am your Witness. I want to represent You.
##### &#x1F5F3; If you like what I do, consider voting for me &#x1F5F3;
[![Vote](https://steemitimages.com/DQmfRx7jNSrFAnRZPrKRHLDxR22gRFYTBnKr2uLNdVty3MG/drakos-witness-thanks.gif)](https://steemit.com/~witnesses)
###### Check my guide about [Voting for Witnesses](https://steemit.com/witness-category/@drakos/tips-and-guidelines-voting-for-witnesses).
###### Go to https://steemit.com/~witnesses. Next to my name, click the https://steemitimages.com/DQmYq3yGqzvbK4JQFudeXoDp6PEtdny8vSvX5swqW5sA2fE/chevron-up.png once.
###### Alternatively you can use SteemConnect to [vote for me](https://app.steemconnect.com/sign/account-witness-vote?witness=drakos&approve=1) or [set me as proxy](https://app.steemconnect.com/sign/account-witness-proxy?proxy=drakos&approve=1)
</center>