Follower template tutorial
This section runs through using the follower template we created in IOTICS Host Library.
Prerequisites
You need to have:
- Created the Follower Template as described in IOTICS Host Library.
- Gone through the Publisher template tutorial
Context
In follower.py the example:
Create a twin representing the follower
def create_follower_twin(self):
twin_id = self.agent_auth.make_twin_id('RandomTempFolllower')
self.twin_api.create_twin(twin_id)
self.follower_twin_id = twin_id
Text based searches for and follows twins
This search will return twins with matching text in their or their feeds' labels, descriptions or tags. You can see the publisher adding such a tag to the twins it creates in its _set_twin_meta method. Once found, these twins have their 'random_temperature_feed' subscribed to by the follower, and a callback is set that will fire whenever data is shared to this feed.
"""Find and follow twins"""
search_resp_gen = None
try:
# search for twins
# note: a generator is returned because responses are yielded
# as they come in asynchronously from the network of hosts
search_resp_gen = self.search_api.search_twins(text='Random')
...
for search_resp in search_resp_gen:
for twin in search_resp.twins:
subscription_id = None
try:
# follow twin's feed
subscription_id = self.follow_api.subscribe_to_feed(
self.follower_twin_id, twin.id.value, 'random_temperature_feed', self.follow_callback
)
Semantic based searches for and follows twins
The generated example searches for twins using a text based search, but you can also search using custom semantic properties. The search in the example code snippet below, will return the twins identified with the Temperature Category (ie, the twins with a 'category' predicate set to 'Temperature', according to the IRIs used below).
You can see how to add semantic meta data in the publisher template section or in the README of the publisher folder.
Note:
It is possible to search for the twins identified with a set of properties (a "and" is performed). Read more about search in the Iotics documentation: How to search.
from iotic.web.rest.client.qapi import ModelProperty, Uri
[...]
def follow_twins(self):
"""Find and follow twins"""
...
try:
# Searching Semantically - run a search for all the twins identified with the Temperature category
temperature_property = ModelProperty(key='http://data.iotics.com/ns/category',
uri_value=Uri(value='http://data.iotics.com/category/Temperature'))
search_resp_gen = self.search_api.search_twins(properties=[temperature_property])
...
# Follow all feeds on all twins in the search result.
found_twins = 0
subscription_count = 0
for search_resp in search_resp_gen:
for twin in search_resp.twins:
found_twins += 1
twin_id = twin.id.value
for feed in twin.feeds:
feed_id = feed.feed.id.value
subscription_id = self.follow_api.subscribe_to_feed(
self.follower_twin_id, twin_id, feed_id, self.follow_callback
)
if subscription_id:
subscription_count += 1
logger.info('Subscribed to feed %s on twin %s', feed_id, twin_id)
logger.info('Found %s twins; subscribed to %s new feeds.', found_twins, subscription_count)
Log the received data
As data is base64-encoded before being shared, it must be decoded before use.
def follow_callback(header, body): # pylint: disable=W0613
decoded_data = base64.b64decode(body.payload.feed_data.data).decode('ascii')
temperature = json.loads(decoded_data)
timestamp = body.payload.feed_data.occurred_at.isoformat()
logger.info('Received temperature data %s at time %s', temperature, timestamp)
Repeatedly search to find and follow new twins
def run(self):
logger.info('Follower started')
self.create_follower_twin()
while True:
self.follow_twins()
logger.info('Sleeping for %s seconds', self.loop_time)
sleep(self.loop_time)
Get feed's most recent data via the InterestApi
Get most recent data is available via the InterestApi if the followed feed has the tag store_last set to True.
See the Publisher template for an example.
def get_most_recent_data(self, followed_twin_id: str, feed_id: str):
""" Get feed's most recent data via the InterestApi
Note: the feed metadata must include store_last=True
"""
logger.info('Get most recent data via InterestApi')
most_recent_data = self.interest_api.get_feed_last_stored(follower_twin_id=self.follower_twin_id,
followed_twin_id=followed_twin_id,
feed_id=feed_id)
decoded_data = base64.b64decode(most_recent_data.feed_data.data).decode()
temperature = json.loads(decoded_data)
logger.info('Most recent data %s', temperature)
In conf.py:
Add a configurable search frequency
update_frequency_seconds: int = 300
Note:
The configuration is used in this template is built using pydantic (see https://pydantic-docs.helpmanual.io/)
As an example of how this configuration works, you can change this update_frequency_seconds value from its default of 300, by setting the environment variable UPDATE_FREQUENCY_SECONDS e.g. to set it to 10 seconds: export UPDATE_FREQUENCY_SECONDS=10.
Tests and checks
Note:
To run the following commands, you need to setup your own Python virtual environment:
python3 -mvenv env
source ./env/bin/activate
pip install -U pip setuptools
make unit-static-tests # Run unit and static tests using tox
make static-tests # Run static tests only, using tox
make unit-tests # Run unit tests only, using tox
Running locally
Create credentials for the component
Run gen_creds.py script from the iotics-host-lib directory with your resolver URL. If you don't know your resolver URL you can find it by going to your IOTICSpace's index page: https://example.iotics.space/index.json
This will generate:
- ** USER_SEED and USER_KEY_NAME together are used to generate your agent DID
- ** AGENT_SEED and AGENT_KEY_NAME together are used to generate your agent DID
# Create a virtual environment:
python3 -mvenv venv
source venv/bin/activate
pip install -U pip setuptools
pip install iotics-identity
# or use an existing environment and then call:
./scripts/gen_creds.py --resolver [resolver url e.g. https://your.resolver]
Note: Answer yes (y) to creating a new user seed, and also yes to creating a new agent seed.
Once the script successfully completes, take a note of variables for your component:
export RESOLVER_HOST=https://your.resolver
export USER_SEED=dec8615d1fc1598ceade592a6d756cad3846d8a2fc9a26af7251df7eb152b771
export USER_KEY_NAME=00
export AGENT_SEED=000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f
export AGENT_KEY_NAME=00
Those should be kept safe and be present in the environment in which you are running your connector. If you are using the same user for a publisher component and a follower component, you can reuse the same USER SEED and USER KEY NAME, but note that these should NOT be stored in production environment with your component, instead keep it safe and secure elsewhere.
Run component
To run the component you need to export the credentials you created during the create credentials step and direct the component to run your against space.
See the example below.
| Variable | Description |
|---|---|
| USER_SEED | Returned during the create credentials step |
| USER_KEY_NAME | Returned during the create credentials step |
| AGENT_SEED | Returned during the create credentials step |
| AGENT_KEY_NAME | Returned during the create credentials step |
| QAPI_URL | The URL for your space's API, based on your space's URL. For example, if your space is https: examplespace.iotics.space then the URL would be: https://examplespace.iotics.space/qapi |
| QAPI_STOMP_URL | The URL for your space's STOMP API calls, based on your space's URL. For example, if your space is https: examplespace.iotics.space then the URL would be: wss://examplespace.iotics.space/ws |
| RESOLVER_HOST | Returned during the create credentials step |
Example
An example of the component being run locally with the example credentials previously generated:
# run locally in the follower template folder
# then export some environment variables
export RESOLVER_HOST=[address of resolver for your space]
export USER_SEED=[user seed from above e.g. lmnop5678...]
export USER_KEY_NAME=[user seed from above e.g. 00]
export AGENT_SEED=[agent seed from above e.g. lmnop5678...]
export AGENT_KEY_NAME=[user seed from above e.g. 00]
export QAPI_URL=[address of qapi]
export QAPI_STOMP_URL=[address of qapi stomp endpoint]
# next either
make docker-run # Run using the docker image
# OR
python3 -mvenv env
source ./env/bin/activate
pip install -U pip setuptools
make setup-dev
make run # Run using the sources from your computer
Note:
The QAPI_URL and QAPI_STOMP_URL can be found by going to your IOTICSpace's index page:
https://example.iotics.space/index.json
Updated over 2 years ago