Python coroutines for Hive

View this thread on: d.buzz | hive.blog | peakd.com | ecency.com
·@felixxx·
0.000 HBD
Python coroutines for Hive
## Context
This is for beginners.
For when your first Python bot or script for Hive goes to production.
I am not the right guy to post about clean code, but I've learned a few things.
I've made some mistakes and will make more, but maybe I can help you avoid some.

https://images.hive.blog/0x0/https://files.peakd.com/file/peakd-hive/felixxx/EorySYrJyMpYZG2zMY4pf5feMJJZ7noKRs8LMyL9LMQ2Fpt4CnuDsvEfMpYoJVSdgHm.png

## Coroutines

[In an earlier post](/@felixxx/your-1st-python-hive-bot-deployment) I demonstrated some deployment strategies for Hive bots.
To improve on those bots, you'll need coroutines.
This has nothing to do with Hive. 
In the following, I demonstrate how I use them.
Python supports 3 methods to handle coroutines.

### multi-processing

This is the most complicated and fastest.
But Python is not fast to begin with. 
If computation was the issue, I'd use a different language entirely.
So multi-processing is out the window - I don't want _complicated_.

### asyncio

I am not the right guy to talk about this and it confused me for a while.
async is different in javascript and Python and that made it more complicated.
_Normal_ Python code is already synchronous, while js in browser sometimes just starts coroutines and doesn't wait... 

Anyways, _normal_ Python codes executes like this:

```
import time

def normal(interval):
    print("normal for:", interval, "s")
    time.sleep(interval)
    print('normal finished')

normal(2)
normal(1)
```
The _normal_ function waits for each part of the procedure and continues line by line.
This means the main thread is blocked when I use time.sleep.
```
normal for: 2 s
normal finished
normal for: 1 s
normal finished
```
Async could be used like this:

```
import asyncio

async def asynced(interval):
    print("async for:", interval, "s")
    await asyncio.sleep(interval)
    print("async finished")

loop = asyncio.new_event_loop()
loop.create_task(asynced(1))
loop.create_task(asynced(3))
loop.run_forever()
```
While one function is ***a***waiting, another task will get executed.
```
async for: 1 s
async for: 3 s
async finished
async finished
```
In javascript in browser, when using await, you kind of want the opposite effect.
Using fetch for example kicks off a coroutine and you get a Promise as result of your fetch.
You need to await or use a callback. That can be confusing.

Anyways, in Python async is also usefull for when you are waiting for an external event.
Because while you are waiting, you can unblock the thread and start a different task.
Also, if one task breaks, the next one will still execute.

## threading

In the example I'll use threading **and** async.
In the script above, the loop.runs_forever. And that still blocks the thread.

```
import time
import asyncio
import threading

async def job(interval):
    print("job for:", interval, "s")
    time.sleep(interval)
    print("job finished")

loop = asyncio.new_event_loop()
threading.Thread(target=loop.run_forever).start()

asyncio.run_coroutine_threadsafe(job(2), loop)

time.sleep(1)
print("program keeps flowing")
```
Now the main thread is unblocked, and you can access the async loop in the separate thread with run_coroutine_threadsafe. I use it a a queue, which I add tasks to.
```
job for: 2 s
program keeps flowing
job finished
```
Even if the job threw an exception and broke, the main thread would keep rolling.
## Complexity

This adds a layer of complexity to Python.
Sequential flow of routines is much easier to handle.
Threadsafety and such things are no joke. Beware of race conditions 😵.
I mean: imagine the async example writing chunks to a file - you'd never be sure in which order they get written. It's a high price to pay...

## Performance

The way I understand it, Python gets compiled and executed in such a way, that in the end it only uses one system thread. So what ever you do with threading here, if it is for performance, I'd suggest looking at other soultions.

On the other hand, async libraries for Python such as aiohttp and uvicorn and stuff use async and make the whole thing fast enough to build a websocket server with.
And with flask or other things like it, you can just let gunicorn spin up 100 Python workers in each their own system thread.

## Application

In the end it all depends on what you want to build.
I assume the next step from building Python scripts is to get to a service, that others can use, too.
I am not sure what to call an _app_ anymore, but I'll try to build one on this blog.

## Conclusion

This post was mainly for @arc7icwolf.
I complained about his while true loop and never showed an alternative.
I'll keep blogging and hope it makes sense...
👍 , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,