/ README.MD
README.MD
1 # Reasonable Planning AI 2 3 [🚀 WE LAUNCHED!](https://www.fab.com/listings/dd0fadd5-172c-4938-ae9c-be527eab8b09) 4 5 Discuss the future of this project with us in real time on our [Discord](https://discord.gg/fKFhN2NwjC) 6 7  8 9 Create an AI that can think for itself and plan a series of actions using only a data driven Editor. Your players will be astounded by the smart responsiveness of the AI to your game environment and marvel at it's ability to adapt and act on the fly! 10 11 Reasonable Planning AI is a drop in solution to provide a robust design and data driven AI that thinks for itself and plans it's actions accordingly. One can create AI with no Blueprints nor C++. Reasonable Planning AI acheives this through the Reasonable Planning AI Editor. A robust Unreal Engine editor that lets you predefine a set of actions, a set of goals, and a state. All logic is achieved through the use of various data driven constructs. These components put together are known as Reasonable Planning AI Composer. 12 13 Reasonable Planning AI is also extensible using either Blueprints or C++. You can opt to extend composer with custom `RpaiComposerActionTasks` or go a pure code route and implement the Core Rpai components. 14 15 Reasonable Planning AI is also cross-compatible with Behavior Trees and can execute AITasks. It also comes with an extension to integrate Composer designed Reasonable Planning AI into an existing Behavior Tree through a pre-defined BTTask node within the plugin. 16 17 Reasonable Planning AI utilizes the visual logger for easier debugging of your AI. Reasonable Planning AI Editor also has a builtin hueristics testing tool. You can define a given starting state, and ending desired state, and visual the goal the AI will select under those conditions as well as the action plan it will execute. 18 19 ## Design of Reasonable Planning AI 20 21 The core design of the Reasonable Planning AI when using the Core components and the Core RpaiBrainComponent follows a very simple path of execution. This path of execution is illustrated in the below flow chart. 22 23  24 25 Text description of the above flowchart. A Start Logic method is invoked. A goal is selected, determined by the current state. If a goal is not determined, then the AI will idle once, else the goal defines it's desired state and it is applied to a planner used to determine an action plan. If a plan is not able to be formulated, then it will attempt a new evaluation of a desired goal. If a plan is found, it will continue to execute the plan until there are no more actions available within the plan or a command to interrupt the execution of the plan is received. After any of those exit conditions, a goal is determined from a current state and the process repeats. 26 27 There are two layers to the Reasonable Planning AI that make it a robust AI solution for your project. These layers are known as **Core** and **Composer**. 28 29 ### Core 30 31 There are five (5) Core components implementing Reasonable Planning AI that builds the foundation of the plugin. 32 33 - RpaiGoalBase 34 - RpaiActionBase 35 - RpaiPlannerBase 36 - RpaiReasonerBase 37 - RpaiBrainComponent 38 39 The main execution engine for Reasonable Planning AI is the `RpaiBrainComponent`. This is a `UBrainComponent` from the `AIModule` used to execute AI logic and interact with the rest of the `AIModule` defined components. The two logic driving classes are `RpaiRpaiReasonerBase` and `RpaiRpaiPlannerBase`. The `RpaiRpaiReasonerBase` class is used to provide implementations for determining goals. Out of the box, there are two implementations provided `RpaiReasoners/RpaiRpaiReasoner_DualUtility` and `RpaiReasoners/RpaiRpaiReasoner_AbsoluteUtility`. Please read the documentation of each class to understand thier capabilities. For goal reasoning there is one provided solution. This solution is `RpaiPlanners/RpaiRpaiPlanner_AStar`. This implementation determines an action plan by using the `RpaiGoalBase::DistanceToCompletion` and `RpaiActionBase::ExecutionWeight` functions for the cost heuristic. 40 41 `RpaiGoalBase` and `RpaiActionBase` are the classes most developers will implement when not using Composer classes to build a data drive AI within the editor. `RpaiGoalBase` provides functions for determining value (commonly referred to as utility) for a given desired outcome. It also provides functions for determining the effort to accomplish the given goal from a given current state. `RpaiActionBase` provides functions for heuristics to calculate the effort to do an action given a state. It also provides methods for execution. These execution methods are the primary drivers for having the AI act. These functions are similar to the UE baked in AI `BTTasks`. 42 43 ### Composer 44 45 The Composer layer is built on top of the Core layer of Reasonable Planning AI. Composer brings the value of Reasonable Planning AI to Game Designers and others without needing to wire Blueprints or write C++. Because Composer is built on top of Core, any Programmers will be able to intergrate into the Composer framework by simply by inheriting from one of the Core classes or from the extend Composer defined classes. The Composer defined classes are listed below. 46 47 - RpaiComposerGoal 48 - RpaiComposerAction 49 - RpaiComposerBrainComponent 50 - RpaiComposerActionTask 51 - RpaiComposerActionTaskBase 52 - RpaiComposerStateQuery 53 - RpaiComposerStateMutator 54 - RpaiComposerDistance 55 - RpaiComposerWeight 56 - RpaiComposerBehavior 57 58 The relationship of these classes to each other are defined below. For additonal details please refer to the documentation of the classes either in the Unreal Engine Editor or C++ comments. Ultimately the classes are used to configure data queries to provide cost, weight, applicability, and mutations during the goal selection and action planning processes. `RpaiActionTasks` are predefined actions your AI can do within the game world. `RpaiComposerBrianComponent` is an extension of `RpaiBrainComponent` that adds a factory method to create goals and actions plans from the defined `RpaiComposerBehavior` data asset. 59 60 - RpaiComposerGoal 61 - RpaiStateQuery 62 - RpaiComposerDistance 63 - RpaiComposerWeight 64 65 - RpaiComposerAction 66 - RpaiComposerStateQuery 67 - RpaiComposerWeight 68 - RpaiComposerStateMutator 69 - RpaiComposerActionTask 70 71 - RpaiComposerBehavior 72 - RpaiComposerAction[] 73 - RpaiComposerGoal[] 74 - RpaiState: SubclassOf 75 76 - RpaiComposerBrainComponent 77 - RpaiComposerBehavior 78 79 To start using Reasonable Planning AI Composer within the Editor, simply create a new DataAsset within your Content folder and select the type RpaiCmposerBehavior. From there you will be able to define and configure your new AI! See below for a simple tutorial. 80 81 ## Versioning 82 83 Reasonable Planning AI releases are versioned. In source code they are tagged commits. The versioning follows the below format (akin to Semantic Versioning) 84 85 **major.minor.patch-d.d-{alpha|beta|gold}** 86 87 The first tuple of `major.minor.patch` is the Reasonable Planning AI version. Explanations of the numbers are as below: 88 89 > *major* Breaking changes were introduced, significant changes to the behavior of functions, and deprecated functions and fields have been removed. 90 91 > *minor* New fields or functions were added. Some fields or functions may be marked as deprecated. No breaking changes introduced. 92 93 > *patch* Bug fixes, no new field additions or functions introduced. May have a change in behavior. 94 95 The second tuple of `d.d` is the version of Unreal Engine this release is compatible with. This means there could be multiple versions of Reasonable Planning AI to indicate UE compatibility. 96 97 The last part is the `{alpha|beta|gold}` indicating the stability of the release. 98 99 > *alpha* builds **DO NOT** honor the `major.minor.patch` promises of changes. These releases compile and pass all tests. Releases marked as `alpha` can be dramatically different in between versions and upgrades are not advised. DO NOT use alpha for your game project or product. 100 101 > *beta* builds are stable releases that are anticapted to be upgraded to `gold`. They meet all requirements and are fully featured. Breaking changes are not anticpated and beta builds will honor the versioning promises. 102 103 > *gold* A production release. Fully featured and ready to go. May be up on the Marketplace (pending approval) 104 105 ## Quick Start Tutorial AI 106 107 You can give Reasonable Planning AI a quick try by configuring a simple AI by following the steps below. The simple AI will have a goal to move towards a target location and an action that involves having the AI walk to that target location. When a Reasonable Planning AI is designed with a 1:1 Goal to Action design, it is the equivalent of creating a Utility AI. This is what this tutorial will create. 108 109 > This works for both Unreal Engine 4.27.2 and Unreal Engine 5.0.2. Screenshots are taken in UE5, but the workflow is the same with the exception of step 1.a where the Place Actors window should already be open in 4.27.2. 110 111 - Create a New Project (Third Person C++) with Starter Content or open an existing project and create a new Basic world with Navigation 112 - In your project select New -> Level. In the dialogue select "Basic". 113 - Place a Navigation Volume from the "Place Actors" panel. If you do not see it, open by selecting Window -> Place Actors. Search for "Nav" and drag and drop "NavMeshBoundsVolume" into the Level Viewport. 114 115  116 117 - Set Location to 0,0,0 118 - Set the Brush X and Y values to 10000.0 and the Z value to 1000.0 119 120  121 122 - To confirm navigation covers the floor, press `P` to visualize the NavMesh. It should have a green overlay. 123 124  125 126 - Create an `RpaiComposerBehavior` 127 - Right click your Content Browser drawer and select Miscelaneous > Data Asset 128 129  130 131 - Search for "Rpai" and select `RpaiComposerBehavior` 132 133  134 135 - Configure your newly named Composer Behavior 136 - For the `ConstructedStateType` select `RpaiState_Map` 137 - For the `Reasoner` select `Rpai Reasoner Dual Utility` 138 - For the `Planner` select `Rpai Planner AStar` 139 140  141 142 - Add a Goal by clicking the plus sign to the right of the `Goals` field. 143 144 > Note within Rpai Vector to Vector comparisons are done via the squared value of the distance between the two vectors. `FVector::DistSquared`. 145 146 - Select the arrow to the left of `Index [ 0 ]` 147 - For `Distance Calculator` select `Rpai Distance State`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary. 148 - For `Right Hand Side State Reference Key`, expand details and set `State Key Name` to "CurrentLocation" and `Expected Value Type` to "Vector" 149 - For `Left Hand Side State Reference Key`, expand details and set `State Key Name` to "TargetLocation" and `Expected Value Type` to "Vector" 150 151  152 153 - For `Weight` select `Rpai Weight Distance`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary. 154 - For `Distance` select `Rpai Distance State`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessar 155 - For `Right Hand Side State Reference Key`, expand details and set `State Key Name` to "CurrentLocation" and `Expected Value Type` to "Vector" 156 - For `Left Hand Side State Reference Key`, expand details and set `State Key Name` to "TargetLocation" and `Expected Value Type` to "Vector" 157 158 > Because there are no other goals configured in this tutorial, the Weight is somewhat of a throw away configuration. Ideally, the weight of a goal represents the value of choosing this goal. 159 160  161 162 - For `Is Applicable Query` select `Rpai State Query Compare Distance Float`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary. 163 - For `Comparison Operation` select "Greater Than" 164 - For `Distance` select `Rpai Distance State`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessar 165 - For `Right Hand Side State Reference Key`, expand details and set `State Key Name` to "CurrentLocation" and `Expected Value Type` to "Vector" 166 - For `Left Hand Side State Reference Key`, expand details and set `State Key Name` to "TargetLocation" and `Expected Value Type` to "Vector" 167 - For `RHS` set the value to "90000.0" (this is 300.0 squared) 168 169  170 171 - For `Is in Desired State Query` select `Rpai State Query Compare Distance Float`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary. Configure these fields the same as the step above. 172 - For `Comparison Operation` select "Less Than Or Equal To" 173 - For `Distance` select `Rpai Distance State`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary 174 - For `Right Hand Side State Reference Key`, expand details and set `State Key Name` to "CurrentLocation" and `Expected Value Type` to "Vector" 175 - For `Left Hand Side State Reference Key`, expand details and set `State Key Name` to "TargetLocation" and `Expected Value Type` to "Vector" 176 - For `RHS` set the value to "90000.0" (this is 300.0 squared) 177 178  179 180 - Set the `Category` to 0 (this is the Highest Priority Group) 181 - Set the `Goal Name` to "TravelToTargetLocation". 182 - Add an Action by clicking the plus sign 183 - For `Weight Algorithm` select `Rpai Weight Distance` 184 - For `Distance` select `Rpai Distance State`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary 185 - For `Right Hand Side State Reference Key`, expand details and set `State Key Name` to "CurrentLocation" and `Expected Value Type` to "Vector" 186 - For `Left Hand Side State Reference Key`, expand details and set `State Key Name` to "TargetLocation" and `Expected Value Type` to "Vector" 187 188  189 190 - For `Action Task` select `Rpai Action Task Move To`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary 191 - Change `Action Task State Key Value Reference` by setting `State Key Name` to "TargetLocation". 192 193  194 195 - For `Apply to State Mutators`, press the plus icon to add a new element and expand using the dropdown arrow on the left side, then expand `Index [ 0 ]` in the same manner as before. 196 - For the element select `Rpai State Mutator Copy State`. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary. 197 - Expand `State Property to Copy` and set `State Key Name` to "TargetLocation" and `Expected Value Type` to "Vector" 198 - Expand `State Property to Mutate` and set `State Key Name` to "CurrentLocation" and `Expected Value Type` to "Vector" 199 200  201 202 > Keep in Mind: The action mutators only have an impact in the hueristics of planning an action plan. Do not think of this as actions happening over time in your game. Rather, consider the state of the AI after the action has fully completed. So for this tutorial, when the action is completed, the AI agent will be at the location of "TargetLocation" (or at least near it). Do not try to make everything exact, fuzzy values work best here. 203 204 - For `Is Applicable Query` set the value to `Rpai State Query Every`. 205 206 > Note: An empty `Rpai State Query Every` is equivalent to an Always True configuration. An empty `Rpai State Query Any` is equivalent to an Always False configuration. 207 208 - Set the `Action Name` to "WalkToTargetLocation" 209 - Save and close your new behavior data asset 210 211  212 213 - Create a child class of `RpaiComposerBrainComponent`. 214 215 > Important Note: At this state the generation of current state does occur in code or Blueprints. This is a temporary stop gap and will be data driven in the future. 216 217  218 219 - Open your newly created Blueprint Class 220 - Hover over Functions and select Override and `Set State from Ai` 221 222  223 224 - Configure the function definition as pictured below 225 226  227 228 - In the "Event Graph" call `Start Logic` in the `Event Begin Play` node. 229 230  231 232 - In the Details panel for your component, expand the `Rpai` section and set the `Reasonable Planning Behavior` field to your newly created `RpaiComposerBehavior` data asset. 233 - Create a new AIController child class. 234 - Add your newly created Brain Component as a component to the newly created AIController class. 235 - Create a new Character Blueprint. 236 - Set the AI Controller class for your new Character Blueprint to your newly created AIController class. 237 - Set the SkeletalMeshComponent Mesh to the Mannequin Mesh 238 - Set the Animation to Use Anim Blueprint and select the Third Person Anim Blueprint (if not already configured) 239 - Place your new AI Character in the World and Press "Play" or "Simulate" (Alt+S on Windows) 240 241  242 243 - Marvel and your AI walks to the defined "TargetLocation"! 244 - Now go out there and create some fascinating AI and shared it on the [Troll Purse Discord](https://discord.com/invite/TGhNfHtS) in the *#trollpurse-oss* channel! 245 246 247 ## Design Patterns 248 249 Porting Utility AI design patterns to Goal selection in Reasonable Planning AI as based on the reasearch found in [neu.edu](https://course.ccs.neu.edu/cs5150f13/readings/dill_designpatterns.pdf). 250 Since planning uses a lot of the same structures, one may also consider adding these patterns to the actions for planning. All of the patterns referenced are the same as described in the referenced whitepaper. 251 252 ### Utility Patterns 253 254 #### Opt Out Pattern 255 256 The opt out pattern is a means to signal a consideration for a goal must not be considered regardless of utility. To accomplish this pattern assign a `StateQuery` to the `IsApplicable` array as those return boolean results. To utilize the concept of a logical and, use the `StateQuery_Every`. 257 258 #### Opt In 259 260 The opt in pattern can be included by using an `RpaiWeight_Select` in the `Weight` configuration. This allows the concept of "only one of those reasons needs to be true in order for the option to be valid." Additionally, it can also be implemented by using a `StateQuery_Any` within the `IsApplicable` configuration. 261 262 ### Execution History Patterns 263 264 #### Commit 265 266 Apply a state variable specific to the action. Then in the `StateMutator` amplify a float value that will be applied to a `UWeight_CurveFloat` that returns a reducing weight value as the float value increases. Use a polynomial to settle down the commitment after repeated uses. 267 268 #### Inertia 269 270 Use a `UWeight_CurveFloat` on your goal. Set a float value within your state that changes over time. Have a shorter plan of actions for the goal. 271 272 #### Is Done 273 274 Many answers to follow this design pattern. One can use a boolean toggle on the state and use a combination of `StateMutator` and `StateQuery` within `IsApplicable`. However that is not a scalable soution and couples many actions with each other via state properties. Rather, just consider how goals and actions are weighed and distance is determined and this will be accomplished via careful planning. 275 276 #### Cooldown 277 278 There is no concept of time natively built into the framework. One could implement this by using a float state value and a history to determine last time a goal was chosen. This is only in consideration of heuristics. `ActionTask_Wait` can implement this as part of a `ActionTask_Sequence`. 279 280 #### Do Once 281 282 Use a boolean value on the state and an `IsApplicable` `StateQuery` to test for this value. Once entered, set to false and the Goal or Action will not be considered. 283 284 #### One-Time Bonus 285 286 A weight based on a depreciating float value on the state can accomplish this pattern. 287 288 #### Repeat Penalty 289 290 A combination of an incrementing integer on the state and a curve can accomplish this. 291 292 # Advanced Topics 293 294 Below you will find a collection of topics going deep into the details and possible implementations of actions within RPAI. 295 296 ## Integrating Smart Objects (Or Any AITask) 297 298 Because Reasonable Planning AI was built with flexibilty in mind and parity with the features offered by Behavior Trees, AITasks are a natural integration supported by the Reasonable Planning AI Composer Action Task class. To add an AI Task Action Task to Reasonable Planning AI, simple extend from the parent class `RpaiActionTask_GameplayTaskBase`. Here is an example in Unreal Engine 5 (UE5) on how to add the [Smart Objects Module](https://docs.unrealengine.com/5.0/en-US/smart-objects-in-unreal-engine---overview/) to your game using Reasonable Planning AI. 299 300 First, follow the instructions in the above link to active Smart Objects in your project. In your Build.cs file add the "SmartObjectModule" (if you haven't already) as a Build Dependency. Then create a C++ class similar to what is defined below. 301 302 ### Header Definition 303 ```C++ 304 #pragma once 305 306 #include "CoreMinimal.h" 307 #include "AI/AITask_UseSmartObject.h" 308 #include "Composer/ActionTasks/RpaiActionTask_GameplayTaskBase.h" 309 #include "MyActionTask_WalkToUseSmartObject.generated.h" 310 311 /** 312 * Navigate to and Use a Smart Object 313 */ 314 UCLASS() 315 class MY_API UMyActionTask_WalkToUseSmartObject : public URpaiActionTask_GameplayTaskBase 316 { 317 GENERATED_BODY() 318 319 protected: 320 virtual void ReceiveStartActionTask_Implementation(AAIController* ActionInstigator, URpaiState* CurrentState, AActor* ActionTargetActor, UWorld* ActionWorld) override; 321 322 UPROPERTY(EditAnywhere, Category = SmartObjects) 323 FGameplayTagQuery ActivityRequirements; 324 325 UPROPERTY(EditAnywhere, Category = SmartObjects) 326 float Radius; 327 }; 328 ``` 329 330 ### Source Implementation 331 ```C++ 332 #include "ActionTask_WalkToUseSmartObject.h" 333 #include "AI/AITask_UseSmartObject.h" 334 #include "AIController.h" 335 #include "GameplayTagAssetInterface.h" 336 #include "SmartObjectSubsystem.h" 337 #include "SmartObjectDefinition.h" 338 339 void UMyActionTask_WalkToUseSmartObject::ReceiveStartActionTask_Implementation(AAIController* ActionInstigator, URpaiState* CurrentState, AActor* ActionTargetActor = nullptr, UWorld* ActionWorld = nullptr) 340 { 341 USmartObjectSubsystem* SOSubsystem = USmartObjectSubsystem::GetCurrent(ActionWorld); 342 if (!SOSubsystem) 343 { 344 CancelActionTask(ActionInstigator, CurrentState, ActionTargetActor, ActionWorld); 345 } 346 347 if (auto AIPawn = ActionInstigator->GetPawn()) 348 { 349 FSmartObjectRequestFilter Filter(ActivityRequirements); 350 Filter.BehaviorDefinitionClass = USmartObjectGameplayBehaviorDefinition::StaticClass(); 351 if (const IGameplayTagAssetInterface* TagsSource = Cast<const IGameplayTagAssetInterface>(AIPawn)) 352 { 353 TagsSource->GetOwnedGameplayTags(Filter.UserTags); 354 } 355 356 auto Location = AIPawn->GetActorLocation(); 357 FSmartObjectRequest Request(FBox(Location, Location).ExpandBy(FVector(Radius), FVector(Radius)), Filter); 358 TArray<FSmartObjectRequestResult> Results; 359 if (SOSubsystem->FindSmartObjects(Request, Results)) 360 { 361 for (const auto& Result : Results) 362 { 363 auto ClaimHandle = SOSubsystem->Claim(Result); 364 if (ClaimHandle.IsValid()) 365 { 366 if (auto SOTask = UAITask::NewAITask<UAITask_UseSmartObject>(*ActionInstigator, *this)) 367 { 368 SOTask->SetClaimHandle(ClaimHandle); 369 SOTask->ReadyForActivation(); 370 StartTask(CurrentState, SOTask); 371 return; 372 } 373 } 374 } 375 } 376 } 377 CancelActionTask(ActionInstigator, CurrentState, ActionTargetActor, ActionWorld); 378 } 379 ``` 380 381 ## Scoped Design of Action and ActionTask function Interfaces 382 383 In the code base you will run across a common function interface for Action and ActionTask which will look something like this: 384 385 ``` 386 virtual void SomeFunctionName(AAIController* ActionInstigator, URpaiState* CurrentState, FRpaiMemorySlice ActionMemory, AActor* ActionTargetActor = nullptr, UWorld* ActionWorld = nullptr); 387 ``` 388 389 This is the standard function interface (which may also include `float DeltaSeconds`) used for runtime execution of Actions and ActionTasks. There is a clear and intended idea of scoping within each of the parameters of the function explained below. This may help you during design of your extensions to RPAI where data should live. 390 391 - `AAIController* ActionInstigator`: The lifetime of this controller object is managed by the lifetime defined by Unreal Engine or the AIModule. It is ideal to include variables and functions in your implementation and cast it to you implementation for your Action and ActionTask implementations. Use this for functions and variables that must persist beyond the planning or execution of RPAI. 392 - `URpaiState* CurrentState`: The lifetime of this object is scoped to the lifetime of an executing plan. If you want to share data across actions scoped to the execution of the plan, put the variable here. You may also use any variables here to assist in planning and goal determination (see tutorial above). Once a plan finishes, the state is reset. 393 - `FRpaiMemorySlice ActionMemory`: For those familiar with Behavior Tree C++ instancing of Tasks, this is a similar construct. For those not familiar, this memory object is used as a generic storage container of arbitrary data that must persist across function calls for the defined Action or ActionTask. The lifetime of this memory is a single execution lifecycle (Start -> Update -> Complete | Cancel) of a defined Action or ActionTask. 394 - `AActor* ActionTargetActor`: Same lifetime scope as `AAIController* ActionInstigator*. Defaults to owned pawn of the AI Controller, but could be any AActor of interest. 395 - `UWorld* ActionWorld`: Same lifetime scope as the `AAIController* ActionInstigator` World. An Action or ActionTask is not guarenteed to execute within the same world scope as the AI agent. Therefore, use this if you want to be sure to execute within World scope of the AI Agent.