Multiple Sessions
Managing multiple concurrent sessions
Single vs Multi-Session Modes
In most development and scripting use cases, having only one session active at a time is sufficient. The challenge comes when productionizing agents.
By default, AgentOps operates in single-session mode. All of the base SDK functions work as expected.
agentops.add_tags(...)
and know that you want to perform the function on the one and only active session.As soon as you create a second session, AgentOps enters Multi-Session Mode. As long as more than one session is active, the base SDK functions will no longer work.
If multiple sessions exist, you are expected to call the function on the relevant session. Ex:
Functions on agentops
will no longer work in multi-session mode
When in multi-session mode:
Entering Multi-Session Mode
Creating more than one session puts the AgentOps Client into multi-session mode.
Single Session Examples
All of these examples show using AgentOps in single session mode
agentops.init()
agentops.end_session(end_state="Success")
agentops.init(auto_start_session=False)
session = agentops.start_session()
session.end_session(end_state="Success")
Multi Session Examples
As soon as you create a second session, the SDK operates in multi-session mode.
session_1 = agentops.init()
session_2 = agentops.start_session()
agentops.init(auto_start_session=False)
session_1 = agentops.start_session()
session_2 = agentops.start_session()
Managing Multiple Sessions
After creating a session, be sure to have the session reference available anywhere where data related to that session is collected.
The Session object has methods as described in the docs page.
Start Sessions
Start a new session with init()
or start_session()
depending on whether or not AgentOps has already been initialized.
session_1 = agentops.init()
session_2 = agentops.start_session()
or
agentops.init(auto_start_session=False)
session_1 = agentops.start_session()
session_2 = agentops.start_session()
Stop Sessions
To stop a currently active session, call end_session()
on the session object.
session = agentops.start_session()
session.end_session()
If you lose access to the session object before calling end_session()
, the session will be marked as Indeterminate
.
Functions on Sessions
All methods are described in the docs page.
These methods must be called on the session object:
session = agentops.start_session()
session.record(Event(...))
Examples
Two Session Script
Create two sessions and perform functions on each
REST API
Create a REST server with fast-api and manage sessions
Assigning LLM Calls
When we have multiple active sessions, it’s impossible for AgentOps to know which session a particular LLM call belongs to without a little help.
To track an LLM call, use session.patch()
import agentops
import openai
session = agentops.start_session()
messages = [{"role": "user", "content": "Hello"}]
response = session.patch(openai.chat.completions.create)(
model="gpt-3.5-turbo",
messages=messages,
temperature=0.5,
)
If you’re using the create function multiple times, you can create a new function with the same method.
observed_create = session.patch(openai.chat.completions.create)
obs_response = observed_create(
model="gpt-3.5-turbo",
messages=messages,
temperature=0.5,
)
If you make an LLM completion call without one of these methods while you currently have more than one active session, a MultiSessionException
will be raised.
Exceptions
MultiSessionException
“If multiple sessions exist, you must use session.function(). Example: session.add_tags(…) instead of agentops.add_tags(…).”
Receiving this exception means that you tried to perform a function on the SDK base, but at runtime had more than one active session.
NoSessionException
A session action was attempted while no session existed on the client.