Roland Bruggmann — Portfolio

Personal Pages

« Back to Main Page

Unreal Engine Plugin: ESB Messaging

Version: v2.0.1
Author: Roland Bruggmann

Description

Screenshot of Plugin Content

Screenshot of Plugin C++ Classes

A game plugin providing with remote control related assets for UE4 integration and runtime interaction. These are mainly Blueprints designed as messaging layer to an Enterprise Service Bus ESB where UE4 acts as message endpoint.

Features

Contents

C++ Classes

Dependencies

The plugin was developed using Visual Studio 2019. It makes use of code and/or assets from other plugins which must also be installed (cp. package diagram):

With using this plugin also engine plugins MediaIOFramework, MediaFrameworkUtilities and WmfMedia are enabled.

Screenshot of Package Diagram

Usage

Use the plugin as project plugin (folder MyProject/Plugins). Add the plugin by downloading and unpackaging an archive or using git clone:

git clone https://github.com/brugr9/ESBMessaging.git

Find the file MyProject /Plugins /ESBMessaging /Config /DefaultESBMessaging.ini and copy-paste its lines to file MyProject /Config /DefaultEngine.ini.

Screenshot of Plugin


Table of Contents

1. Messaging Concept

1.1. Message Transport

The plugin implements the ZeroMQ PUB/SUB Pattern from Plugin ‘ZeroMQ’. E.g., the demo map uses a communication channel as follows:

Communication Diagram ‘ESBEvent’:

Communication Diagram ESBEvent

1.2. Interfaces

The messaging layer defines four interfaces: EsbTransform, EsbRenderProperty, EsbTextRender and EsbMediaPlayer.

1.2.1. Interface EsbTransform

Function declarations:

Screenshot of Interface EsbTransform functions and Blueprint overrides:

Screenshot of Interface EsbTransform

1.2.2. Interface EsbRenderProperty

Function declarations:

Screenshot of Interface EsbRenderProperty functions and Blueprint overrides:

Screenshot of Interface EsbRenderProperty

1.2.3. Interface EsbTextRender

Function declarations:

Screenshot of Interface EsbTextRender functions and Blueprint overrides:

Screenshot of Interface EsbTextRender

1.2.4. Interface EsbMediaPlayer

Function declarations:

Screenshot of Interface EsbMediaPlayer functions and Blueprint overrides:

Screenshot of Interface EsbMediaPlayer

1.3. JSON Message

Interface messages are declared as JSON Schema version 7.0, to be found in folder Testing/Schemas: UE4-ESBMessage-v1_0.schema.json.

1.3.1. Interface EsbTransform

Examples:

{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetTransformQuat",
        "Parameter": {
            "Location": {
                "X": 1.0,
                "Y": -3.0,
                "Z": 2.0
            },
            "Rotation": {
                "X": 0.1,
                "Y": 0.2,
                "Z": 0.3,
                "W": 1.0
            },
            "Scale": {
                "X": 0.024,
                "Y": 0.024,
                "Z": 0.024
            }
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetTransform",
        "Parameter": {
            "Location": {
                "X": 1.0,
                "Y": -3.0,
                "Z": 2.0
            },
            "Rotation": {
                "X": 0.0,
                "Y": 0.0,
                "Z": 0.0
            },
            "Scale": {
                "X": 1.0,
                "Y": 1.0,
                "Z": 1.0
            }
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetScale",
        "Parameter": {
            "Scale": {
                "X": 1.0,
                "Y": 2.0,
                "Z": 1.0
            }
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetRotationQuat",
        "Parameter": {
            "Rotation": {
                "X": 0.5,
                "Y": 0.6,
                "Z": 0.7,
                "W": 1.0
            }
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetRotation",
        "Parameter": {
            "Rotation": {
                "X": 45.0,
                "Y": 0.0,
                "Z": 0.0
            }}}}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetLocationAndRotation",
        "Parameter": {
            "Location": {
                "X": 1.0,
                "Y": -3.0,
                "Z": 2.0
            },
            "Rotation": {
                "X": 0.0,
                "Y": 0.0,
                "Z": 0.0,
                "W": 1.0
            }}}}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetLocation",
        "Parameter": {
            "Location": {
                "X": 1.1,
                "Y": -3.2,
                "Z": 1.7
            }}}}

1.3.2. Interface EsbRenderProperty

Examples:

{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetVisibility",
        "Parameter": {
            "Visibility": true
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetMaterial",
        "Parameter": {
            "Material": "PupilLabsCalibrationMarker"
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetColor",
        "Parameter": {
            "Color": {
                "R": 0.5,
                "G": 1.0,
                "B": 0.0,
                "A": 1.0
            }
        }
    }
}

1.3.3. Interface EsbTextRender

Examples:

{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetText",
        "Parameter": {
            "Text": "Unreal Engine Integration by ESB Messaging"
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetFormatText",
        "Parameter": {
            "HorizontalAlignment": "Center",
            "VerticalAlignment": "Center",
            "FontSize": 26
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "SetFontColor",
        "Parameter": {
            "FontColor": {
                "R": 0.5,
                "G": 1.0,
                "B": 0.0,
                "A": 1.0
            }
        }
    }
}

1.3.4. Interface EsbMediaPlayer

Examples:

{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "OpenFile",
        "Parameter": {
            "Path": "C:\\Users\\MyUser\\Documents\\Unreal Projects\\MyProject\\Plugins\\ESBMessaging\\Testing\\Media\\LedScreenTestPattern.mp4"
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "Play",
        "Parameter": {}
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "Pause",
        "Parameter": {
            "PlaybackTime": "00T00:00:03.000"
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "Seek",
        "Parameter": {
            "PlaybackTime": "00T00:00:03.000"
        }
    }
}
{
    "ObjectName": "DemoActor",
    "Function": {
        "FunctionName": "Close",
        "Parameter": {}
    }
}

1.3.5. Interface Result Messages

Also interface result messages are declared as JSON Schema version 7.0, to be found in folder Testing/Schemas: UE4-ESBMessageResult-v1_0.schema.json.

Examples:

{
      "topic": "Watchdog_SetVisibility_Success",
      "payload":
      {
            "call":
            {
                  "objectName": "Watchdog",
                  "functionName": "SetVisibility"
            },
            "result":
            {
                  "returnValue": "Success",
                  "info": "",
                  "dateTimeUTC": "2021.05.27-14.42.59",
                  "millisecond": 145
            }
      }
}
{
      "topic": "DemoActor_SetTransform_Success",
      "payload":
      {
            "call":
            {
                  "objectName": "DemoActor",
                  "functionName": "SetTransform"
            },
            "result":
            {
                  "returnValue": "Success",
                  "info": "",
                  "dateTimeUTC": "2021.05.27-14.43.32",
                  "millisecond": 377
            }
      }
}
{
      "topic": "DemoActor_SetTransformQuat_Success",
      "payload":
      {
            "call":
            {
                  "objectName": "DemoActor",
                  "functionName": "SetTransformQuat"
            },
            "result":
            {
                  "returnValue": "Success",
                  "info": "Function executed.",
                  "dateTimeUTC": "2021.06.17-11.25.50",
                  "millisecond": 225,
                  "pupilHitResult":
                  {
                        "eyeLocation":
                        {
                              "x": 0,
                              "y": -0,
                              "z": 0
                        },
                        "eyeRotation":
                        {
                              "x": 0.014739572070538998,
                              "y": 0.012784205377101898,
                              "z": 0.38275140523910522,
                              "w": 0.9236452579498291
                        },
                        "isHit": true,
                        "hitResult":
                        {
                              "bBlockingHit": true,
                              "bStartPenetrating": true,
                              "faceIndex": -1,
                              "time": 0,
                              "distance": 0,
                              "location":
                              {
                                    "x": 0,
                                    "y": -0,
                                    "z": 0.5
                              },
                              "impactPoint":
                              {
                                    "x": -10,
                                    "y": 10,
                                    "z": 9.9182125268271193e-07
                              },
                              "normal":
                              {
                                    "x": 0,
                                    "y": -0,
                                    "z": 1
                              },
                              "impactNormal":
                              {
                                    "x": 0,
                                    "y": -0,
                                    "z": 1
                              },
                              "traceStart":
                              {
                                    "x": 0,
                                    "y": -0,
                                    "z": 0.5
                              },
                              "traceEnd":
                              {
                                    "x": 0,
                                    "y": -0,
                                    "z": 0
                              },
                              "penetrationDepth": 4950,
                              "item": -1,
                              "elementIndex": 0,
                              "physMaterial": "None",
                              "actor": "StaticMeshActor'/ESBMessaging/Demo/Maps/UEDPIE_0_Map_ESB_Demo.Map_ESB_Demo:PersistentLevel.Plane_2'",
                              "component": "StaticMeshComponent'/ESBMessaging/Demo/Maps/UEDPIE_0_Map_ESB_Demo.Map_ESB_Demo:PersistentLevel.Plane_2.StaticMeshComponent0'",
                              "boneName": "None",
                              "myBoneName": "None"
                        }
                  }
            }
      }
}

1.4 ESB Message Handler Component

An ESBMessaging interface function may be called by handing over a JSON-ObjectString message to the EbsMessageHandlerComponent by calling function HandleMessage which deserialises the message and calls the desired function. Sequence by function call HandleMessage:

  1. Deserialize message which is assumed to be a JSON-ObjectString to a JSON-Object.
  2. In JSON-Object find field ObjectName, if value equals to ‘EbsMessageHandlerComponent’ > Details Panel > ESBMessaging > ‘Handled Object Name’ (Default value: ‘DemoActor’): continue processing.
  3. In JSON-Object find and branch on field FunctionName and call an interface bundled evaluation method.
  4. In the interface bundled evaluation method switch on FunctionName.
  5. Within switch-case statement: in JSON-Object find field Parameter, process them to UE values, do some transformation eventually (RHS-to-LHS or unit scale) and pass the values calling a function declared by interface.

Screenshot EsbMessageHandlerComponent BlueprintCallable Function Node HandleMessage:

Screenshot EsbMessageHandlerComponent BlueprintCallable Function Node 'HandleMessage'

The component provides also with BlueprintCallable interface bundled function nodes to serialize interface result messages as JSON-ObjectString:

Screenshot EsbMessageHandlerComponent BlueprintCallable Serialize Function Nodes:

Screenshot EsbMessageHandlerComponent BlueprintCallable Serialize Function Nodes

Furthermore the component provides with BlueprintReadOnly EsbMessageHandlerMessages (cp. screenshot):

Screenshot EsbMessageHandlerComponent BlueprintReadOnly EsbMessageHandlerMessages

1.5. ESB Message Handler Actor

An EsbMessageHandlerActor:

EsbMessageHandlerActor inheriting classes:

Screenshot Blueprint EsbMessageHandlerActor:

Screenshot Blueprint EsbMessageHandlerActor

Class Diagram EsbMessageHandlerActor:

Class Diagram EsbMessageHandlerActor

The event-dispatchers are implemented as dynamic multicast delegates which can be bound several times at the same time (multicast) and may be used in C++ as well as in Blueprints, both (dynamic) (cp. table ‘Overview Event-Dispatcher Macros’):

Overview Event-Dispatcher Macros

1.5.1. ESB TextRender Actor

An EsbTextRenderActor:

1.5.1.1. UE4 TextRender Alignment

Change of horizontal and/or vertical alignment results in a visually moving center of the text as the alignment is in relation to the TextRender pivot (cp. screenshots below).

Screenshots of TextRender: 1) horizontal alignment ‘left’ combined with vertical alignment ‘bottom’ (default); 2) horizontal alignment ‘center’ combined with vertical alignment ‘center’; 3) horizontal alignment ‘right’ combined with vertical alignment ‘top’:

Screenshot of TextRender horizontal alignment 'left' combined with vertical alignment 'bottom' Screenshot of TextRender horizontal alignment 'center' combined with vertical alignment 'center' Screenshot of TextRender horizontal alignment 'right' combined with vertical alignment 'top'

1.5.2. ESB MediaPlayer Actor

An EsbMediaPlayerActor:

1.5.2.1. UE4 MediaPlayer State Machine

Behavior of UE4 MediaPlayer:

Transition table:

State Diagram ‘MediaPlayer’:

State Diagram 'MediaPlayer'

1.6. ESB Message Handler Child Actor

An EsbMessageHandlerChildActor:

EsbMessageHandlerChildActor inheriting classes:

Screenshot Blueprint EsbMessageHandlerChildActor:

Screenshot Blueprint EsbMessageHandlerChildActor

1.6.1. ESB Pupil Gaze Actor

An EsbPupilGazeActor:

Screenshot Blueprint EsbPupilGazeActor:

Screenshot Blueprint EsbPupilGazeActor

Class Diagram EsbPupilGazeActor:

Class Diagram EsbPupilGazeActor

1.6.1.1. Trace for Hit Generation

On EsbPupilGazeActor calling function SetLocationAndRotation, SetTransformQuat, SetTransform, SetLocation, SetRotation or SetRotationQuat, the APupilGazeActor private function Trace is triggered which performs a ray-tracing trying to generate a hit (channel visibility) and updates following ChildActorComponent values:

In case of a hit the objects are updated using values from the hit-result. If no hit was obtained, the objects are updated with values from the trace end.

2. Plugin Settings and BP Function-Library

2.1. Plugin Settings

Screenshot Plugin Settings:

Screenshot of Plugin Settings

2.2. BP Function-Library

The ESB Blueprint Function-Library provides with functions to access the plugin settings values as follows:

Screenshot Blueprint Function-Library:

Screenshot of Blueprint Function-Library

3. Blueprints

3.1. BP ESB Watchdog

Screenshot Blueprint BP_ESB_Watchdog:

Screenshot Blueprint BP_ESB_Watchdog

3.2. BP ESB MediaPlayer2D

Please consider:

Screenshot Blueprint BP_ESB_MediaPlayer2D:

Screenshot Blueprint BP_ESB_MediaPlayer2D

3.3. BP ESB MediaPlayer360

Please consider:

Screenshot Blueprint BP_ESB_MediaPlayer360:

Screenshot Blueprint BP_ESB_MediaPlayer360

3.4. BP ESB TextRender

Please consider:

Screenshot Blueprint BP_ESB_TextRender:

Screenshot Blueprint BP_ESB_TextRender

3.5. BP ESB BlackShieldBack

Screenshot Blueprint BP_ESB_BlackShieldBack:

Screenshot Blueprint BP_ESB_BlackShieldBack

3.6. BP ESB BlackShieldFloor

3.7. BP ESB BlackShieldFront

3.8. BP ESB BlackShieldLeft

3.9. BP ESB BlackShieldRight

3.10. BP ESB ClearColor

Please consider:

Screenshot Blueprint BP_ESB_ClearColor:

Screenshot Blueprint BP_ESB_ClearColor

3.11. BP ESB CollisionWarning

Screenshot Blueprint BP_ESB_CollisionWarning:

Screenshot Blueprint BP_ESB_CollisionWarning

3.12. BP ESB Head

Screenshot Blueprint BP_ESB_Head:

Screenshot Blueprint BP_ESB_Head

3.13. BP ESB PupilGaze0

Please consider:

Screenshot Blueprint BP_ESB_PupilGaze0:

Screenshot Blueprint BP_ESB_PupilGaze0

3.14. BP ESB PupilGaze0Ray

Please consider:

Screenshot Blueprint BP_ESB_PupilGaze0Ray:

Screenshot Blueprint BP_ESB_PupilGaze0Ray

3.15. BP ESB PupilGaze0Hit

Please consider:

Screenshot Blueprint BP_ESB_PupilGaze0Hit:

Screenshot Blueprint BP_ESB_PupilGaze0Hit

3.16. BP ESB PupilGaze1

Please consider:

3.17. BP ESB PupilGaze1Ray

Please consider:

3.18. BP ESB PupilGaze1Hit

Please consider:

3.19. BP ESB PupilShapeTarget

Please consider:

Screenshot Blueprint BP_ESB_PupilShapeTarget:

Screenshot Blueprint BP_ESB_PupilShapeTarget

4. Testing

4.1. Demo Actor and Demo Map

Blueprint ESBMessaging Demo Actor BP_ESB_DemoActor:

Screenshot of Blueprint ‘BP_ESB_DemoActor’

Screenshot of BP_ESB_DemoActor

Map Map_ESB_Demo, World Outliner:

Screenshot of Map_ESB_Demo:

Screenshot of Map_ESB_Demo

In the ESBMessaging demo map Map_ESB_Demo LevelBlueprint the ZeroMQ-Sockets of ZmqSubSocketActor and ZmqPubSocketActor are bound. Finally a timer triggered event starts to call ZmqSubSocketActor function Receive every millisecond.

Screenshot of Map_ESB_Demo LevelBlueprint:

Screenshot of Map_ESB_Demo LevelBlueprint

4.2. Packaging

Brief instruction: Packaging, e.g., a shipping build (cp. Packaging Projects):

  1. Project Settings > Project > Maps & Modes > Default Maps > Game Default Map: Map_ESB_Demo
  2. File > Package Project > Build Configuration > Shipping
  3. File > Package Project > Windows (64-bit)
  4. Point to a folder of your choice

In the folder of your choice a folder ‘WindowsNoEditor’ is created containing the executable file (*.exe).

4.3. Jupyter Notebook

For testing purpose this plugin provides with Jupyter Notebooks:

Please find instructions on how to install Anaconda and setup Jupyter Notebook in folder Testing.

Screenshot of Jupyter Notebook ESBMessaging ZeroMQ Publish:

Screenshot of Jupyter Notebook ESBMessaging ZeroMQ Publish

A. References

Enterprise Service Bus ESB

JSON Schema Specification

Unreal Engine Documentation

« Back to Main Page