Skip to main content

Battle Session

1. What's Battle Session​

Battle session is the foundation of battle service. Battle session instances can be created through services such as Lobby / Matchmaking / World or via the PGOS HTTP API, and placed in on-line or local DS via the DS Hosting service. The battle session also manages data such as the players in the game and the DS access information.

NOTE

PGOS allow players to join multiple battle-sessions at the same time. Game client can use the API DescribeBattleSessions to query the battle-sessions the player has participated in.

2. Life cycle of a Battle Session​

2.1 Creation​

There are two ways to create a battle session in PGOS in summary:

  1. Through battle services such as Lobby / Matchmaking / World.
    • After the Lobby owner calls the StartBattle interface, the Lobby service will request the creation of a battle session from the Battle Session Management service.
    • The Matchmaking service will request the creation of a battle session from the Battle Session Management service after completing the aggregation of a set of match tickets.
    • The World will request the creation of a battle session from the Battle Session Management service based on player responsibility.
  2. Through the PGOS HTTP API StartBattleSessionPlacement. You can apply for a battle session directly to the battle session management service without going through a battle service.

For the complete creation process of a battle session, refer to the DS Hosting integration document of the DS Hosting service.

NOTE

Battle Session Management is an internal service within PGOS, which is not exposed to users. However, knowing the existence of this service helps users understand the working mechanism of the Battle Service.

2.2 Activation​

Once a battle session instance is created, it is handed over to an available DS resource (which could be an on-line DS process running on a CVM or a local DS process running on the developer's local machine). The DS can choose to activate the battle session immediately or after completing the preparations to welcome players.

This process is referred to as battle session placement in PGOS. The event of a battle session being activated is notified to the game client through events in the Lobby / Matchmaking / World modules of PGOS, to inform the game client that the DS is ready.

PGOS pushes the status changes of the battle session to all members in the game, so that the clients are aware that the DS running the game is connectable, specifically when the battle session is activated.

2.3 Termination​

Once a battle session is terminated, it cannot be re-enabled (activated). Currently, there are several ways to terminate a battle session:

  • DS can destroy its held battle session at any time by calling the ProcessEnding interface. It is recommended to properly handle the players within the game before performing this operation.
  • Operations personnel can manually terminate an ongoing battle session in the PGOS Portal. This is usually used as an emergency measure in exceptional circumstances or to handle those battle sessions that have not terminated as scheduled during development and debugging.
  • The World service supports the automatic recycling of idle battle sessions (i.e., those without players).

2.4 State Transition​

stateDiagram-v2 [*] --> Pending: start a battle session placement Pending --> Active: DS called ActivateBattleSesssion() Pending --> Timeout: ActivateBattleSesssion() didn't called Active --> Terminated: battle session ends Terminated --> [*] Timeout --> [*]
  • Pending to Active: DS can trigger this state transition by calling the interface ActivateBattleSession
  • Pending to Timeout: This status transition will be triggered if the DS does not call ActivateBattleSession within a limited time
  • Active to Terminated: DS can trigger this state transition by calling the interface ProcessEnding

3. Player Battle Session​

Battle session maintains a session for each player be assigned to the battle what we called player battle session. These sessions are initialized immediately when the battle session is activated by game DS, and will be different hen a player is assigned to the same battle session multiple times.

Status transitions of player battle session:

stateDiagram-v2 [*] --> Pending Pending --> Reserved: ReservePBS [*] --> Reserved Pending --> Completed: RemovePBS Reserved --> Active: AcceptPBS Reserved --> Completed: RemovePBS Active --> Completed: RemovePBS Active --> Disconnected: DisconnectPBS Disconnected --> Active: AcceptPBS Disconnected --> Completed: RemovePBS Completed --> [*]
  • Pending: This state is only used in the world service. When a player requests to join a locked battle session, a battle session in the Pending state will be assigned so that DS can further process its joining request.
    • DS can call the ReservedPlayerBattleSession interface to reserve a seat for a player in the game.
    • DS can also call RemovePlayerBattleSession to remove a player from a battle session.
  • Reserved: All player battle sessions turn to Reserved when the battle session is activated by DS
  • Reserved to Active: Only DS can trigger this state transition by calling the interface AcceptPlayerBattleSession. Games can use this interface to verify the legitimacy of player identities with the PGOS backend. We recommend that DS make this a required step to process player connection requests.
  • Pending / Reserved / Active / Disconnected to Completed: DS can end a player battle session by calling the interface RemovePlayerBattleSession. Client can also call the interface TerminatedPlayerBattleSession to end a player battle session. Once pbs reaches the Completed state, calling the AcceptPlayerBattleSession interface on it will return failure.
  • Active to Disconnected: DS can trigger this state transition by calling the interface DisconnectPlayerBattleSession. The Disconnect status is only used to help the game mark that the player is disconnected at this time, and has nothing to do with the actual connection status between the player and the DS.
  • Disconnected to Active: DS can trigger this state transition by calling the interface AcceptPlayerBattleSession.

4. Inspect Battle Session on Portal​

4.1 Query Battle Sessions From Battle Session Logs​

Open web portal, select sub menu Battle Session bellow Battle menu. You can use the following parameters to search for battle sessions:

  • Fleet. The DS fleet that battle sessions located in.
  • Config. The match/lobby configuration that produced battle sessions.
  • ID. Specify a battle session id. Battle session id could be obtained either from the matchmaking result on client side or from FDelegateOnStartBattleSession event on DS side.
  • Status. The status of battle sessions.
  • Created Time. The created time of battle sessions.

image-20220607194347829

Export Searched Result: You can export the searched results to a CSV file, of which PGOS supports a maximum of 5,000 pieces of data for export. If the data is greater than 5,000, only the first 5,000 pieces of data will be exported.

View Battle Session Details: Click on Battle Session ID in the Battle Session searching result to view detailed information.

4.2 Query Battle Sessions on A Specified Fleet​

Another way to query the battle session under the specified fleet is: open web portal, select sub menu Fleets bellow DS & Builds menu, then select Battle Sessions tab.

1631517935782

Click on Battle Session ID in the Battle Session searching result to view detailed information.

4.3 Player Battle Sessions​

Battle session maintains a player battle session for each player be assigned to the battle. These player battle sessions are initialized immediately when the battle session is activated by game DS.

Note:

We maintain a set of data for each player battle session to help you tracking the players in the battle. This information can be viewed in the details page of battle session.

1631519416863

  • Player Battle Session ID: The identifier assigned to the player battle session.
  • Player: Display name of the player. Click to view detail info of the player.
  • Status: Player battle session status. Read here to learn how to manage player battle sessions on game DS
    • Reserved - The Player battle session has been reserved, but the player has not yet connected.
    • Active - Means that the player is connected to the game DS.
    • Completed - Player battle session comes to the end. Means that the player is no longer connected.
    • Disconnected - Means that the player is disconnected from the game DS.
  • Uptime: Total length of time the player has been active in the player battle session.
  • Connect Time: Date indicated when the player connected to game server.
  • Terminate Time: Date indicated when the player disconnected from game server.

5. Access Battle Session with PGOS API​

5.1 Access Battle Sessions with Client SDK​

You can use the following interfaces to access, query, and manage the battle session that the player participate in game client.

InterfaceDescription
DescribePlayerBattleSession[DEPRECATED!!!] Declared deprecated, consider using DescribeBattleSessions instead.
DescribeBattleSessionsGet a list of the battle sessions that the player is participating in.
TerminatePlayerBattleSessionTry to terminate the player's battle session., After the calling is successful, players will no longer be able to enter this battle session, please use with caution

UPgosBattleAPI::DescribeBattleSessions

This interface returns the activate player battle sessions for the current player. At the same time return the access information of game DS. Call this interface to obtain the necessary access information if your client closed unexpectedly and want to reconnect to the previous battle.

Interface prototype:

  /**
  * Get the player's battle session info.
  *
  * @param PlayerId The player id
  */
  void DescribeBattleSessions(
        TFunction<void(const FPgosResult& Ret, const FClientPlayerBattleSessionInfo * Data)> Callback) const;

Example Code:

  #include "PgosSDKCpp.h"
  #include "Core/PgosErrorCode.h"

  void SomeUObjectClass::SomeFunction()
  {
     auto Battle = IPgosSDKCpp::Get().GetClientBattleAPI();
     if (Battle)
     {
         FString PlayerId;
         Battle->DescribeBattleSessions([](const FPgosResult& Ret, const DescribeBattleSessionsResult * Data) {
             if (Ret.err_code == (int32)Pgos::PgosErrCode::kSuccess)
             {
                 UE_LOG(LogTemp, Log, TEXT("DescribeBattleSessions Success"));
             }
             else
             {
                 UE_LOG(LogTemp, Log, TEXT("DescribeBattleSessions Failed: err_code=%d, err_msg=%s"), Ret.err_code, *Ret.msg);
             }
         });
     }
  }

UPgosBattleAPI::TerminatePlayerBattleSession

This interface will force change the status of a player battle session to Completed. Calling this interface will trigger a callback to PGOS Server SDK: UPgosHostingAPI::OnPlayerBattleSessionsTermianted, to informs the game DS these players who have left the battle session.

Interface prototype:

  /**
  * Try to terminate the player's battle session.
  * After the calling is successful, players will no longer be able to enter this battle session, please use with caution
  *
  * @param BattleSessionId The battle session id
  * @param PlayerBattleSessionId The player battle session id
  */
  void TerminatePlayerBattleSession(
        const FString& BattleSessionId,
        const FString& PlayerBattleSessionId,
        TFunction<void(const FPgosResult& Ret)> Callback) const;

Example Code:

  #include "PgosSDKCpp.h"
  #include "Core/PgosErrorCode.h"

  void SomeUObjectClass::SomeFunction()
  {
     auto Battle = IPgosSDKCpp::Get().GetClientBattleAPI();
     if (Battle)
     {

        FString BattleSessionId;
        FString PlayerBattleSessionId;
         Battle->TerminatePlayerBattleSession(BattleSessionId, PlayerBattleSessionId, [](const FPgosResult& Ret) {
             if (Ret.err_code == (int32)Pgos::PgosErrCode::kSuccess)
             {
                 UE_LOG(LogTemp, Log, TEXT("TerminatePlayerBattleSession Success"));
             }
             else
             {
                 UE_LOG(LogTemp, Log, TEXT("TerminatePlayerBattleSession Failed: err_code=%d, err_msg=%s"), Ret.err_code, *Ret.msg);
             }
         });
     }
  }

5.2 Access Battle Session with Server SDK​

The game server needs to pay attention to the following core events related to battle session management:

EventDescription
OnHealthCheckThe HealthCheck event will be triggered every one minute to obtain the health status of the game process.
OnStartBattleSessionThe StartBattleSession event will be triggered when a battle session was placed to the server. Game server needs to call the ActivateBattleSession interface within 5 minutes to active battle session, otherwise the battle session will be abandoned.
OnBattleSessionUpdatedThis event is used to notify new players of joining a battle session in two specific scenarios: World and Matchmaking backfill.
OnProcessTerminateThe event will be triggered once the process being terminated. The ProcessTerminate event will be triggered when shrink occurs or someone terminate a battle session manually.
OnPlayerBattleSessionsTerminatedThis event is called periodically to notify game DS of those player battle sessions that have been terminated from client side.

Interfaces below could used to manage battle session and the players in battle session:

InterfaceDescription
ActivateBattleSessionThe API will turn battle session status to Active.
Signals PGOS that the server process is now ready to receive player sessions. Should be called once all battle session initialization has finished.
ProcessEndingSignals PGOS that the process is ending, and the battle session running on this process will also be terminated with a Completed status.
ReservePlayerBattleSessionCall this API to turn player battle session status from Pending to Reserved.
AcceptPlayerBattleSessionValidates a player session, and signal PGOS to update player session's status to Active. This method should be called when a client requests a connection to the server.
DisconnectPlayerBattleSessionMark a player battle session as disconnected. You can call this interface when the player disconnects from DS non-permanently. The session can be resumed at any time by calling the AcceptPlayerBattleSession interface again.
RemovePlayerBattleSessionThis interface will set the player session to a final state(Completed) and cannot be restored. You should call this interface when a player is disconnects from DS permanently.
DescribePlayerBattleSessionsQuery latest players of a battle session.

5.3 Access Battle Session with HTTP API​

PGOS provides several HTTP APIs related to battle management, which assist in initiating battle session placement requests without the need to integrate with services like Lobby / Matchmaking / World. This mode of access is called the Standalone working mode in DS Hosting within PGOS.

Please read the document below for detailed information about DS Hosting Standalone:

Please refer to the document below for related HTTP API:

6. Suggestions on Managing Players in a Battle Session​

This chapter lists several typical scenarios that how players leave the battle session. Aim to help you understand the management of player battle sessions. Please follow the game mechanics during development.

PGOS strictly limits each player to only one active battle session at the same time, so understanding the management of player battle session in a battle session helps you provide players with the on-line gaming experience as you expect.

6.1 Fully Managed by DS​

That is, the status of player battle sessions is fully managed by the game DS. Means that until the game DS calls the interface RemovePlayerBattleSession, the player CAN NOT enter any other battle sessions.

This solution is suitable for games that want to strictly restrict players from exiting the ongoing battle session, but it also has a negative impact on the player experience: The player seems to have lost some freedom anyway.

The general game flows using this solution could as follows:

  • The player is removed from the battle session immediately after disconnecting from the game DS

    sequenceDiagram participant DS2 participant Client participant DS participant PGOS ServerSDK Note left of Client: Player join a battle session Client->>DS: Connect DS->>PGOS ServerSDK: AcceptPlayerBattleSession Note left of Client: Player disconnect DS-->>Client: Disconnect DS->>PGOS ServerSDK: RemovePlayerBattleSession Note left of Client:Player could join a new battle session Client->>DS2: Connect

  • Keep the player battle session for the player till the battle session comes to the end.

    sequenceDiagram participant Client participant DS participant PGOS ServerSDK Note left of Client: Player join a battle session Client->>DS: Connect DS->>PGOS ServerSDK: AcceptPlayerBattleSession Note left of Client: Player disconnect DS-->>Client: Disconnect DS->>PGOS ServerSDK: DisconnectPlayerBattleSession Note left of Client:Player reconnect to the DS Client->>DS: Connect DS->>PGOS ServerSDK: AcceptPlayerBattleSession Note left of DS: Battle ending DS->>PGOS ServerSDK: Remove all players by calling RemovePlayerBattleSession

6.2 Allow the Client to Leave a Battle Session​

In this solution, the player can leave an ongoing battle session by calling the TerminatePlayerBattleSession interface in the client. No matter if the client is already logged in to the game DS at the time.

Handing over the right to leave from the ongoing battle session to the player can prevent the player from being blocked by a bad battle, but it will have a negative impact on other players in the battle. The pros and cons need to be weighed by the developer.

The general game flows using this solution could as follows:

sequenceDiagram participant DS2 participant Client participant DS participant PGOS ServerSDK Note left of Client: Player join a battle session Client->>DS: Connect DS->>PGOS ServerSDK: AcceptPlayerBattleSession Note left of Client: Player disconnect DS-->>Client: Disconnect DS->>PGOS ServerSDK: DisconnectPlayerBattleSession Note left of Client:Player could leave the battle Client->>Client: TermaintePlayerBattleSession(player_id_x) PGOS ServerSDK->>DS:OnPlayerBattleSessionTerminated[player_id_x] Note left of Client:Player could join a new battle session Client->>DS2: Connect

6.3 Check the Ongoing Battle Session for Player​

We strongly recommend that you call the DescribePlayerBattleSession interface after the game client launched to confirm if the player has a ongoing battles session information. Because if there is a ongoing battle session and the player’ s player battle session is not Completed, the player cannot enter other battles. If so, roughly player have two choices:

  • Leave the battle session by calling TerminatePlayerBattleSession interface and then explore more funs.

    sequenceDiagram participant PGOS ClientSDK participant Client participant DS Client->>PGOS ClientSDK: Client launched Client->>PGOS ClientSDK: DescribePlayerBattleSession Note left of Client:Leave current battle Client->>PGOS ClientSDK: TerminatePlayerBattleSession

  • Reconnect to DS to continue the ongoing battle session.

    sequenceDiagram participant PGOS ClientSDK participant Client participant DS participant PGOS ServerSDK Client->>PGOS ClientSDK: Client launched Client->>PGOS ClientSDK: DescribePlayerBattleSession Note left of Client:Continue current battle Client->>DS: Connect DS->>PGOS ServerSDK: AcceptPlayerBattleSession

a

2. Architecture Diagram​

Game client accesses the battle session service through the PGOS Client SDK.

  • Read here to learn how to initiate a battle session through matchmaking.
  • Read here to learn how to initiate a battle session through lobby.

Game DS accesses the battle session service through the PGOS Server SDK.

  • Read here to learn how to manage the life cycle of a battle session, and players in the battle session.

1631516746552

7. Battle Data​

Battle data is designed to build battle history info that could access from game client. The game could customize its own battle history feature with battle data.

Related documents: an typical use case of battle data here.

7.1 Integration Steps​

The game could accesses battle data service through PGOS Server SDK and PGOS Client SDK.

Through PGOS Server SDK, the game could：

  • Build battle data for a specified battle session in key-value format.
  • Set tags in string format for battle data.

Through PGOS Client SDK, the game could:

  • Query battle data list for a specified player and filter by tags and keys.
  • Query battle data for specified battle session id and filter by keys.

1605606131176

7.2 Concepts​

  • Battle Data. A set of key-value pairs persistent data binded to a battle session id, usually used to record data such as battle attribute information, players' information, battle statistics information, etc.

  • Battle Record. PGOS saved all battle sesison ids that players have participated in. Even if the game doesn't write any battle data, it's still possiable to get information about the battle sessions for players through the interface provided by PGOS.

  • Brief & Detail. We use 'brief' and 'detail' as reserved keys to store Brief & Detail info of a battle. A set of wrapped interfaces are provided to access these keys. Of course, these interfaces are optional, and the game could build its own battle data to deal with complex scenarios.

    In wrapped interfaces, we abstract the following two concepts:

    • Brief. A string value for key 'brief'. Generally, we suggest the game use Brief info to storage short and highly concise information, such as winner team, started time, kill/dead rate, etc.
    • Detail. A string value for key 'detail'. Generally, we suggest the game use Detail info to storage long and complex information, such as statistics for each player. Detail info is not necessary if Brief info is enough for the game.

    Concepts diagram:

    1646121114802

Remark

Binary string is not supported for Battle Data key or value

7.3 Build Key-Value Based Battle Data​

Remark

PGOS Server SDK only.

Build battle data for a battle session with battle data service.

  1. Add key-value pairs info to a battle's battle data by calling UpdateBattleData

    • BattleSessionID. The battle session id.
    • RecordData. Custom key-value pairs.

    Call UpdateBattleData from Battle module as follows:

    #include "PgosSDKCpp.h"
    #include "Core/PgosErrorCode.h"

    void SomeUObjectClass::UpdateBattleData()
    {
         auto battle = IPgosSDKCpp::Get().GetServerBattleAPI();
         if (battle)
         {
             FString BattleSessionID;
             TMap<FString, FString> RecordData;

             battle->UpdateBattleData(BattleSessionID, RecordData, [](const FPgosResult& Ret) {
                 if (Ret.err_code == (int32)Pgos::PgosErrCode::kSuccess)
                 {
                     UE_LOG(LogTemp, Log, TEXT("UpdateBattleDataSuccess"));
                 }
                 else
                 {
                     UE_LOG(LogTemp, Log, TEXT("UpdateBattleDataFailed: err_code=%d, err_msg=%s"), Ret.err_code, *Ret.msg);
                 }
             });
         }
    }

  2. Remove keys from a battle's battle data by calling DeleteBattleData

    • BattleSessionID. Battle session id.
    • Keys. Battle data keys to remove.

    Call DeleteBattleData from Battle module as follows:

    #include "PgosSDKCpp.h"
    #include "Core/PgosErrorCode.h"

    void SomeUObjectClass::DeleteBattleData()
    {
         auto battle = IPgosSDKCpp::Get().GetServerBattleAPI();
         if (battle)
         {
             FString BattleSessionID;
             TArray<FString> Keys;

             battle->DeleteBattleData(BattleSessionID, Keys, [](const FPgosResult& Ret, const FServerDelBattleDataResult* Data) {
                 if (Ret.err_code == (int32)Pgos::PgosErrCode::kSuccess)
                 {
                     UE_LOG(LogTemp, Log, TEXT("DeleteBattleDataSuccess"));
                 }
                 else
                 {
                     UE_LOG(LogTemp, Log, TEXT("DeleteBattleDataFailed: err_code=%d, err_msg=%s"), Ret.err_code, *Ret.msg);
                 }
             });
         }
    }

7.4 Set Battle Tags​

Remark

PGOS Server SDK only.

Set tags for a battle session by calling.

  • BattleSessionID. Battle session id of which to build battle history data.
  • Tags. Tags of the battle history record. Mainly used to filter items when querying battle history records from game client.

Call UpdateBattleRecord from Battle module as follows:

#include "PgosSDKCpp.h"
#include "Core/PgosErrorCode.h"

void SomeUObjectClass::UpdateBattleRecord()
{
    auto battle = IPgosSDKCpp::Get().GetServerBattleAPI();
    if (battle)
    {
        FString BattleSessionID;
        TArray<FString> PlayerIDs;
        TArray<FString> Tags;

        battle->UpdateBattleRecord(BattleSessionID, PlayerIDs, Tags, [](const FPgosResult& Ret) {
            if (Ret.err_code == (int32)Pgos::PgosErrCode::kSuccess)
            {
                UE_LOG(LogTemp, Log, TEXT("UpdateBattleRecordSuccess"));
            }
            else
            {
                UE_LOG(LogTemp, Log, TEXT("UpdateBattleRecordFailed: err_code=%d, err_msg=%s"), Ret.err_code, *Ret.msg);
            }
        });
    }
}
tip

player_ids in api UpdateBattleRecord is deprecated from v0.15.0

7.5 Query Battle Records​

Remark

PGOS Client SDK only.

Calling GetBattleRecordList to batch get battle records for player. You would get all key-value pairs of each battle record, which were set by UpdateBattleData from game server.

  • PlayerID. The player id to query.
  • Tags. Get the list for specific tags, if the tags is empty, the tag will not be filtered.
  • Keys. Get the list for specific keys, if the keys is empty, the key will not be filtered.
  • Offset. The starting position of the list.
  • Count. The count of the list.

Call GetBattleRecordList from Battle module as follows:

#include "PgosSDKCpp.h"
#include "Core/PgosErrorCode.h"

void SomeUObjectClass::GetBattleRecordList()
{
    auto battle = IPgosSDKCpp::Get().GetClientBattleAPI();
    if (battle)
    {
        FString PlayerID;
        TArray<FString> Tags;
        TArray<FString> Keys;
        int32 Offset;
        int32 Count;

        battle->GetBattleRecordList(PlayerID, Tags, Keys, Offset, Count, [](const FPgosResult& Ret, const FClientBattleListInfo* Data) {
            if (Ret.err_code == (int32)Pgos::PgosErrCode::kSuccess)
            {
                UE_LOG(LogTemp, Log, TEXT("GetBattleRecordListSuccess"));
            }
            else
            {
                UE_LOG(LogTemp, Log, TEXT("GetBattleRecordListFailed: err_code=%d, err_msg=%s"), Ret.err_code, *Ret.msg);
            }
        });
    }
}

7.6 Query Battle Data​

Remark

PGOS Client SDK only.

Calling GetBattleData to batch get battle data for a specific battle session. You would get specified key-value pairs of each battle data, which were set by UpdateBattleData from game server.

  • BattleSessionID. The id to query.
  • Keys. Get the list for specific keys, if the keys is empty, the key will not be filtered.

Call GetBattleData from Battle module as follows:

#include "PgosSDKCpp.h"
#include "Core/PgosErrorCode.h"

void SomeUObjectClass::GetBattleData()
{
    auto battle = IPgosSDKCpp::Get().GetClientBattleAPI();
    if (battle)
    {
        FString BattleSessionID;
        TArray<FString> Keys;

        battle->GetBattleData(BattleSessionID, Keys, [](const FPgosResult& Ret, const FClientBattleDataInfo* Data) {
            if (Ret.err_code == (int32)Pgos::PgosErrCode::kSuccess)
            {
                UE_LOG(LogTemp, Log, TEXT("GetBattleDataSuccess"));
            }
            else
            {
                UE_LOG(LogTemp, Log, TEXT("GetBattleDataSuccess: err_code=%d, err_msg=%s"), Ret.err_code, *Ret.msg);
            }
        });
    }
}

8. Key Errors Handling​

Error CodeRelevant APIHandling Suggestion
kBackendNeedRetryActivateBattleSession
HeartbeatBattleSession
TerminateBattleSession
AcceptPlayerBattleSession
RemovePlayerBattleSession
DisconnectPlayerBattleSession		It means that the operation on battlesession failed. The concurrency safety is guaranteed by distributed lock mechanism. It's recommended to retry for limited times until it succeeds.