Ctrl+K

client.connector

client.connector#

The connector module provides connectors that abstract the client communication with Redis.

class soulsai.distributed.client.connector.DQNConnector(config: SimpleNamespace)#

The DQN client connector abstracts the communication with the training server.

The connector sends samples and episode info messages into a message queue. A separate message consumer process is responsible to consume the messages and upload them to Redis. This allows the main script to avoid blocking uploads between environment steps.

In addition, an update notification process checks if a new model is available, and a model update process downloads the new weights. The main client therefore never explicitly changes its network. Instead, the new network weights are loaded during training. For this reason, the agent and transformations have to share their memory with the update process.

Warning

In order to avoid races during the update or a mismatch in model IDs and network weights, these attributes should always be accessed by using the connector as context manager.

Example

con = DQNConnector(config)
with con:
    con.agent(obs)
    print(con.model_id)

If the communication with Redis is interrupted, the connector will automatically try to reestablish the connection. In the client thread, this is only visible through a logger warning. After reestablishing the connection, sample and episode info messages will be sent again, and model updates will resume.

property model_id: int#

Model ID.

Always use with the context manager! See DQNConnector.__enter__().

push_sample(sample: bytes)#

Send a sample message over the message queue.

Parameters:

sample – Experience sample.

push_episode_info(episode_info: bytes)#

Send an episode info summary message over the message queue.

Parameters:

episode_info – Episode info dictionary.

close()#

Close the connector by stopping the message consumer, updater and heartbeat process.

class soulsai.distributed.client.connector.PPOConnector(config: SimpleNamespace)#

The PPO client connector abstracts the communication with the training server.

The connector sends samples and episode end messages into a pipe. A separate message consumer process is responsible to consume the messages and upload them to Redis. This allows the main script to avoid blocking uploads between environment steps.

After the client has reached the required number of samples, it synchronizes with the server to update its model.

push_sample(sample: bytes)#

Send a sample message over the message pipe.

Parameters:

sample – Experience sample. Also contains the model ID and step ID.

push_episode_info(episode_info: bytes)#

Send an episode info message over the message pipe.

Parameters:

episode_info – Episode info dictionary.

close()#

Close the connector by stopping the message consumer and heartbeat process.

sync(timeout: float = 100.0)#

Wait for the server to update its network weights and load them into the client agent.

Note

This function should only be called after all the required samples have been sent to the server. Otherwise, the training step on the server is never triggered.

Parameters:

timeout – Maximum waiting time for synchronization.

Raises:

ServerTimeoutError – The server did not respond with an update within the timeout period.

property model_id: int#

Model ID.

Always use with the context manager! See DQNConnector.__enter__().

Contents