Running Hivemind & HAfAH on HAF + Jussi - 2023
hive-139531ยท@mahdiyariยท
0.000 HBDRunning Hivemind & HAfAH on HAF + Jussi - 2023
<center>https://www.youtube.com/watch?v=vOV2srXzNW4</center><center><sub>*stolen video from gtg*</sub></center> Related post: [How to run a HAF node](/hive-139531/@mahdiyari/how-to-run-a-haf-node-2023) *** Table of contents: - PostgreSQL configs for hivemind - Hivemind on HAF - HAfAH (account history) - Jussi *** I assume you are here after following the [previous post](/hive-139531/@mahdiyari/how-to-run-a-haf-node-2023). The process is usually the same for production and development. ### Preparing PostgreSQL Add the following line to the file `/pg/workdir/haf-datadir/haf_postgresql_conf.d/custom_postgres.conf` ``` hba_file = '/home/hived/datadir/haf_postgresql_conf.d/custom_pg_hba.conf' ``` **Do NOT change the path!** That path is for inside the docker container. You might have to create the above file. That will allow us to override the default pg_hba.conf for PostgreSQL and allow authorization for user haf_admin. Create the following file and add the following lines: `/pg/workdir/haf-datadir/haf_postgresql_conf.d/custom_pg_hba.conf` ``` host haf_block_log haf_admin 172.0.0.0/8 trust host haf_block_log haf_app_admin 172.0.0.0/8 trust # DO NOT DISABLE! # If you change this first entry you will need to make sure that the # database superuser can access the database using some other method. # Noninteractive access to all databases is required during automatic # maintenance (custom daily cronjobs, replication, and similar tasks). # # Database administrative login by Unix domain socket local all postgres peer # TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all all peer # IPv4 local connections: host all all 127.0.0.1/32 md5 # IPv6 local connections: host all all ::1/128 md5 # Allow replication connections from localhost, by a user with the # replication privilege. local replication all peer host replication all 127.0.0.1/32 md5 host replication all ::1/128 md5 ``` This ๐ is the default pg_hba included in HAF with the addition of the first line. Restart the HAF container and you should be good to go. ``` docker stop haf-instance cd /pg/workdir ../haf/scripts/run_hived_img.sh registry.gitlab.syncad.com/hive/haf/instance:instance-v1.27.4.0 --name=haf-instance --data-dir=$(pwd)/haf-datadir --shared-file-dir=/dev/shm --detach ``` This will allow access to the database as `haf_admin` which is needed for preparing the database for hivemind. You can optionally remove only the first line afterwards. ### Hivemind on HAF Clone hivemind: ```bash cd /pg git clone https://gitlab.syncad.com/hive/hivemind cd hivemind git checkout v1.27.4.0.0 git submodule update --init --recursive ``` Building ```bash cd /pg/workdir ../hivemind/scripts/ci/build_instance.sh v1.27.4.0.0 ../hivemind registry.gitlab.syncad.com/hive/hivemind/ ``` Preparing the database ```bash sudo apt install postgresql-client -y ../hivemind/scripts/setup_postgres.sh --postgres-url=postgresql://haf_app_admin@172.17.0.2/haf_block_log ../hivemind/scripts/setup_db.sh --postgres-url=postgresql://haf_admin@172.17.0.2/haf_block_log ``` Finally we can run the hivemind. You will have to start two instances one as a sync and another as a server instance to serve the APIs. Sync: ```bash ../hivemind/scripts/run_instance.sh registry.gitlab.syncad.com/hive/hivemind/instance:v1.27.4.0.0 sync --database-url="postgresql://haf_app_admin@172.17.0.2:5432/haf_block_log" --docker-option=--detach --docker-option=--name=hivemind-sync ``` Check logs ``` docker logs hivemind-sync -f --tail 50 ``` Server: ```bash ../hivemind/scripts/run_instance.sh registry.gitlab.syncad.com/hive/hivemind/instance:v1.27.4.0.0 server --database-url="postgresql://haf_app_admin@172.17.0.2:5432/haf_block_log" --docker-option=--detach --docker-option=--name=hivemind-server ``` Check logs ```bash docker logs hivemind-server -f --tail 50 ``` Hivemind is very slow at syncing and will probably take 3-4 days depending on your storage & CPU speed. *** ### HAfAH - Account history on HAF Running HAfAH is easy as it doesn't need to sync. It will use the data already present in HAF. ```bash cd /pg git clone https://gitlab.syncad.com/hive/HAfAH cd HAfAH git checkout v1.27.4.0.0 git submodule update --init --recursive ``` Building ```bash # inside HAfAH folder scripts/ci-helpers/build_instance.sh "v1.27.4.0.0" . registry.gitlab.syncad.com/hive/hafah --haf-postgres-url=postgresql://haf_app_admin@172.17.0.2:5432/haf_block_log ``` Running ```bash docker run --rm -itd --name=hafah-instance registry.gitlab.syncad.com/hive/hafah/instance:instance-v1.27.4.0.0 ``` Check logs ```bash docker logs hafah-instance -f --tail 50 ``` ### Jussi Jussi is used as a proxy which exposes all the APIs and handles caching and timing out certain APIs. First we need to get the IP address of hivemind-server and hafah-instance. ``` docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' haf-instance # 172.17.0.2 docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' hivemind-server # 172.17.0.4 docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' hafah-instance # 172.17.0.5 ``` Clone jussi ``` cd /pg git clone https://gitlab.syncad.com/hive/jussi cd jussi ``` Edit `Dockerfile` and put `#` in front of or remove the following at line 116 ``` # RUN pipenv run pytest ``` If your IP addresses are different, replace them in the config below. Add following lines in `config.json` ``` { "limits": { "accounts_blacklist": [""] }, "upstreams": [ { "name": "hived", "translate_to_appbase": true, "urls": [["hived", "http://172.17.0.2:8090"]], "ttls": [ ["hived", 3], ["hived.login_api", -1], ["hived.network_broadcast_api", -1], ["hived.market_history_api", 1], ["hived.database_api", 3], ["hived.database_api.get_block", -2], ["hived.database_api.get_block_header", -2], ["hived.database_api.get_content", 1], ["hived.database_api.get_dynamic_global_properties", 1] ], "timeouts": [ ["hived", 5], ["hived.network_broadcast_api", 0] ] }, { "name": "appbase", "urls": [ ["appbase.wallet_bridge_api", "http://172.17.0.2:8090"], ["appbase.condenser_api.get_account_reputations", "http://172.17.0.4:8080"], ["appbase.follow_api.get_account_reputations", "http://172.17.0.4:8080"], ["appbase.condenser_api.broadcast_transaction", "http://172.17.0.2:8090"], ["appbase.network_broadcast_api", "http://172.17.0.2:8090"], ["appbase.block_api.get_block", "http://172.17.0.2:8090"], ["appbase.block_api.get_block_range", "http://172.17.0.2:8090"], ["appbase.condenser_api.get_block", "http://172.17.0.2:8090"], ["appbase.condenser_api.get_accounts", "http://172.17.0.2:8090"], ["appbase.condenser_api.get_active_votes", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_blog", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_blog_entries", "http://172.17.0.4:8080"], [ "appbase.condenser_api.get_comment_discussions_by_payout", "http://172.17.0.4:8080" ], ["appbase.condenser_api.get_content", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_content_replies", "http://172.17.0.4:8080"], [ "appbase.condenser_api.get_discussions_by_author_before_date", "http://172.17.0.4:8080" ], ["appbase.condenser_api.get_discussions_by_blog", "http://172.17.0.4:8080"], [ "appbase.condenser_api.get_discussions_by_comments", "http://172.17.0.4:8080" ], ["appbase.condenser_api.get_discussions_by_created", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_discussions_by_feed", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_discussions_by_hot", "http://172.17.0.4:8080"], [ "appbase.condenser_api.get_discussions_by_promoted", "http://172.17.0.4:8080" ], [ "appbase.condenser_api.get_discussions_by_trending", "http://172.17.0.4:8080" ], ["appbase.condenser_api.get_follow_count", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_followers", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_following", "http://172.17.0.4:8080"], [ "appbase.condenser_api.get_post_discussions_by_payout", "http://172.17.0.4:8080" ], ["appbase.condenser_api.get_reblogged_by", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_replies_by_last_update", "http://172.17.0.4:8080"], ["appbase.condenser_api.get_trending_tags", "http://172.17.0.4:8080"], ["appbase.database_api.list_comments", "http://172.17.0.4:8080"], ["appbase.database_api.list_votes", "http://172.17.0.4:8080"], ["appbase.database_api.find_votes", "http://172.17.0.4:8080"], ["appbase.database_api.find_comments", "http://172.17.0.4:8080"], ["appbase.tags_api.get_discussion", "http://172.17.0.4:8080"], ["appbase.account_history_api", "http://172.17.0.5:6543"], ["account_history_api", "http://172.17.0.5:6543"], ["appbase.account_history_api.get_ops_in_block", "http://172.17.0.5:6543"], ["appbase.account_history_api.enum_virtual_ops", "http://172.17.0.5:6543"], ["appbase.account_history_api.get_transaction", "http://172.17.0.5:6543"], ["appbase.account_history_api.get_account_history", "http://172.17.0.5:6543"], ["condenser_api.get_ops_in_block", "http://172.17.0.5:6543"], ["condenser_api.get_transaction", "http://172.17.0.5:6543"], ["condenser_api.get_account_history", "http://172.17.0.5:6543"], ["appbase.condenser_api.get_ops_in_block", "http://172.17.0.5:6543"], ["appbase.condenser_api.get_transaction", "http://172.17.0.5:6543"], ["appbase.condenser_api.get_account_history", "http://172.17.0.5:6543"], ["database_api.get_account_history", "http://172.17.0.5:6543"], ["appbase.database_api.get_account_history", "http://172.17.0.5:6543"], ["appbase", "http://172.17.0.2:8090"] ], "ttls": [ ["appbase", 1], ["appbase.block_api", -2], ["appbase.block_api.get_block_range", -1], ["appbase.database_api", 1], ["appbase.condenser_api.get_account_reputations", 3600], ["appbase.condenser_api.get_ticker", 1], ["appbase.condenser_api.get_accounts", 3], ["appbase.condenser_api.get_account_history", 3], ["appbase.condenser_api.get_content", 3], ["appbase.condenser_api.get_profile", 3], ["appbase.database_api.find_accounts", 3], ["appbase.condenser_api.get_dynamic_global_properties", 1], ["appbase.condenser_api.get_ops_in_block.params=[2889020,false]", 0], [ "appbase.account_history_api.get_ops_in_block.params={\"block_num\":2889020,\"only_virtual\":false}", 0 ] ], "timeouts": [ ["appbase", 5], ["appbase.network_broadcast_api", 0], ["appbase.condenser_api.broadcast_block", 0], ["appbase.condenser_api.broadcast_transaction", 0], ["appbase.condenser_api.get_ops_in_block.params=[2889020,false]", 20], [ "appbase.account_history_api.get_ops_in_block.params={\"block_num\":2889020,\"only_virtual\":false}", 20 ], ["appbase.condenser_api.get_account_history", 20] ] }, { "name": "hive", "translate_to_appbase": false, "urls": [["hive", "http://172.17.0.4:8080"]], "ttls": [["hive", -1]], "timeouts": [["hive", 30]] }, { "name": "bridge", "translate_to_appbase": false, "urls": [["bridge", "http://172.17.0.4:8080"]], "ttls": [ ["bridge", -1], ["bridge.get_discussion", 3], ["bridge.get_account_posts", 3], ["bridge.get_ranked_posts", 3], ["bridge.get_profile", 3], ["bridge.get_community", 3], ["bridge.get_post", 3], ["bridge.get_trending_topics", 3] ], "timeouts": [["bridge", 30]] } ] } ``` Building ``` docker build -t jussi . ``` Running jussi at 0.0.0.0:80 ``` # Run where config.json is located # Don't change the path docker run --rm -itd --env JUSSI_UPSTREAM_CONFIG_FILE=/app/config.json -v $(pwd)/config.json:/app/config.json -p 80:8080 --name jussi-instance jussi ``` To run in local network, replace `80:8080` with `127.0.0.1:80:8080` Check the logs ``` docker logs jussi-instance -f --tail 50 ``` That's it. You should be able to access all the APIs at port 80 of your public IP address. *** The official GitLab repositories *might* include more information: https://gitlab.syncad.com/hive/hivemind https://gitlab.syncad.com/hive/HAfAH https://gitlab.syncad.com/hive/jussi *** Related post: [How to run a HAF node](/hive-139531/@mahdiyari/how-to-run-a-haf-node-2023) Feel free to ask anything. *** <center>  </center>
๐ gaottantacinque, not-a-gamer, cribbio, din0baba, gasaeightyfive, leduc1984, cambridgeport90, jsalvage, smartvote, zeero22, onestop, ghici, lothbrox, ibook-ishare, psyo, gurujames, dappcast, sensiblecast, sunnyjolly, jhiecortez, hijosdelhombre, trentonlundy1, underground, rezawijaya, foodtaster, luvlylady, bai123, vonzeppelin, irisworld, natha93, supperbotsps.com, riyuuhi, makrotheblack, elizabeth-22, abduljalill, sward, dreemsteem, virgo27, amico.sports, mahdiyari, apartphilosopher, steemwhalepower, bjornb, davidbright, biggerjoe, kgcoin, sorenkierkegard, walidchabir, emjoe, maclevis, owasola, artmusiclife, tonybad, gejzep, ihsan19, repayme4568, macmaniac77, khoon, jkp.nisha, mengao, apostle-thomas, polashsen, tm2909, bahagia-arbi, ond, kim24, archangel21, carbodexkim, victoraraguayan1, adysscheryl, eliasseth, islandboi, overunitydotcom, djblendah, houstonrockets, unlockmaster, disruptivas, nnn1jls, marco77290, qaiserch0078, dumitriu, roleerob, elricmoonslayer, glstech, azirgraff, cuthamza, mrxplicit, gakimov, nitin21, mirnasahara, marcocasario, vsemmetall, raghib, chike4545, hive-defender, cryptoshotsdoom, tibra, dec.entralized, elcomentador, key-defender.shh, adiiba, cloh76, digital.mine, mayib, edundayo, tillmea, sarix, daniel2001, tinyhousecryptos, sannae.art, kitsuki, xpax120, foxlatam, joeyarnoldvn, hugo1954, patagonian-nomad, freebornsociety, tesaganewton, ekkah, steveconnor, currentxchange, mangos, dreamon, rustam-02, achmadyani, baroen96, sirmiraculous, joanpablo, karelia, aquarius.academy, jff7777, lynliss, shanibeer, sbi6, sbi-tokens, gerber, thedailysneak, mice-k, dcityrewards, ezzy, exyle, deathwing, g4fun, zackarie, steem.leo, someguy123, lloyddavis, one.life, rafalski, bobby.madagascar, determine, milu-the-dog, leo.syndication, dpend.active, sketching, jelly-cz, hivechat, photo-hive-five, leodis, daan, shitsignals, jeanlucsr, felander, unconditionalove, olgadmitriewna, bestboom, themightyvolcano, dlike, permaculturedude, haikusailor, lividseagulls, therealyme, dcrops, alphacore, blockbrothers, emrebeyler, nateaguila, cakemonster, samsemilia7, goodcontentbot, xyz004, steemindian, kgswallet, triplea.bot, invest2learn, ribary, hykss.leo, carmary, josewilchez, netaterra, veryanprime, steinz, hungryharish, hungryanu, mrhoofman, dpoll.witness, drexlord, lordb, nfaith, firinmahlazer, dickturpin, liable, holoferncro, bdlatif, forecasteem, bucipuci, rituraz17, offoodandart, beverages, the-table, gmlrecordz, chungsu1, lightflares, vicesrus, throwbackthurs, cryptological, fractalfrank, hornetsnest, leone.bianco, gooze, horpey, dbfoodbank, jaybird, weddinggift, hive-data, rudy-dj, sirjaxxy, gringo211985, justasperm, solarwarrior, nazaleo, lays, felixgarciap, gsbilbao, youraverageguy, laiyuehta, heyitshaas, coquiunlimited, enjar, tasteofsuccess, steem-tube, ilovegames, akpos, steeminer4up, ycgg, pladozero, goodcontentbot1, samuraiscam, hairyfairy, teamashen, contentisking, radiv, netzisde, creepycreature, culturearise369, luciancovaci, kharrazi, blockchaincat, leomolina, curatorwhale, dipoabasch, kont82, prapanth, iccel35, hhaskana, barslanhan, nipek, oomcie85, hyborian-strain, kriptoqraf, themarkymark, cloudhyip, buildawhale, makerhacks, upmyvote, apeminingclub, gniksivart, steemseph, kingsofa, recoveryinc, samrisso, blockheadgames, hivewp, movement19, santinhos, sunisa, recan, abduljalil.mbo, willsaldeno, britcoins, kimchi-king, sankysanket18, afrikablr, monstervoter, gocular, yildiss, crescendoofpeace, patlu, silberpapst, sitiaishah, estourefugiado, wakeupnd, abieikram, penguinpablo, cryptonized, hungrybear, funnyman, jacuzzi, teachblogger, seikatsumkt, nasel, anutta, freshfund, herbncrypto, plebtv, gvincentjosephm, princeso, fuzzythumb, marcost4, bryanj4, ian19, kshahrck, bolsfit, leguidecrypto, lynbabe10, theleapingkoala, srl-zone, uruiamme, danielsaori, bilpcoin.pay, steembet.asia, acidyo, diosarich, adamada, worldwildflora, xves, juanvegetarian, ghaazi, rendrianarma, leveluplifestyle, trasto, mdasein, mister-meeseeks, jazzhero, didutza, sgbonus, wannderingsoul, creodas, manniman, i-test-stuff, ninnu, fitnessgourmet, patlog, razorshark, marvinman, money.finance, mannimanccadm, nyxlabs, elyelma, riki1, meltysquid, anonymous02, traciyork, nathen007, betterdev, rishi556, giftgiver, hextech, mafia.wallet, nftmart, h-e, thecouncil, rishi556.engine, cadawg, foxon, self-help.dev, treefrognada, steemflow, patsitivity, marito74, appreciator, detlev, dynamicrypto, bertrayo, braaiboy, manuelmusic, alenox, discoveringarni, lucianav, elgatoshawua, naath, aprasad2325, beauty197, aslamrer, bluemist, sunsea, samostically, outlinez, ravenmus1c, leighscotford, iamjohn, gabilan55, syberia, noalys, trouvaille, power-kappe, mariaser, fotomaglys, yoieuqudniram, tiyumtaba, sovebrito, artefactoestudio, robtheranger, incubot, afternoondrinks, ilanisnapshots, emeka4, wynella, inciter, noelyss, coolsurfer, kattycrochet, thereikiforest, hjrrodriguez, auracraft, trostparadox, beerlover, gubbatv, gubbahomestead, xyba, taskmaster4450, unyimeetuk, rpren, the.rocket.panda, marianosdeath, tzae, mitchcabz, johndieo, wiseagent, bitcoinflood, trafalgar, itinerantph, raindrop, traf, kgsupport, julesquirin, el-dee-are-es, nthtv, aarauz15, valerianis, aftersound, davicoscz, wend1go, thisnewgirl, olaunlimited, rodolfomandolina, mukadder, doriantaylor, w74, hannes-stoffel, ausbitbank, drbanner, santigs, softworld, farm1, gigel2, irfanmustafa, photographercr, gabrielatravels, franzpaulie, workaholic666, privatblog, asgarth, rubelynmacion, borislavzlatanov, violator101, deepresearch, saltycat, travoved, onealfa.vyb, onealfa, rxhector, rival, nikoleondas, funshee, davdiprossimo, shebe, besheda, deeanndmathews, hope.venezuela, hive-122727, phortun, onlavu, wrestlingdesires, vyb.curation, dwinblood, richardcrill, nonsowrites, dalz.shorts, flamistan, khoola, trostparadox.vyb, crazygirl777, leprechaun, anacristinasilva, steemexperience, vyb.pob, topbooster, coffeebuds, proofofbrainio, d-guardian, saboin.pob, mineopoly, mcsagel, artjohn, swifty6ix, plusvault, pobscholarship, vyb.fund, awuahbenjamin, nrg, ubg, sophieandhenrik, tatiana21, v4vapid, kgakakillerg, primeradue, proymet, benainouna, isabelpena, resonator, ura-soul, informationwar, risemultiversity, dmwh, empress-eremmy, deepdives, inigo-montoya-jr, delver, investinfreedom, tftproject, truthforce, haccolong, riskneutral, shanhenry, kennyskitchen, stevescoins, dbroze, hoaithu, anhvu, drag33, queercoin, henrietta27, amakauz, caelum1infernum, hivedeb, chidistickz, insanityisfree, atma.love, newsnownorthwest, marilui91, hiddendragon, commonlaw, clownworld, louis88, ssebasv, jimbobbill, farpetrad, oflyhigh, wherein, adm, opinizeunltd, zaibkang, a-quarius, melcaya, amiegeoffrey, fizz0, foodchunk, ikrahch, lordneroo, spectrumecons, vegoutt-travel, captainhive, russia-btc, kreativita, jphamer1, sharkthelion, hikergirl, elikast, anaivelazco31, olebulls, lisfabian, chinito, spiritabsolute, brutus22, orionsbeltbuckle, phabulu, rammargarita, dgoyzuetab, john0928, whatsup, arabisouri, ahmedsy, ksjegw, irionet, diehardknocks, aswita, albro, newsflash, thelastofcat, revisesociology, bpcvoter3, revise.spk, liverpool-fan, chainableit, samuelstifen, essejparr, jaybone, jayrent, ace108, bbphotography, jagged, tasco111, profwhitetower,