Using SITL with AirSim

AirSim is a simulator for drones, cars and more, built on Unreal Engine (they also have an experimental Unity release, but right now it’s hasn’t been implemented with ArduPilot)

It is open-source, cross-platform and provides excellent physically and visually realistic simulations. It has been developed to become a platform for AI research to experiment with deep learning, computer vision and reinforcement learning algorithms for autonomous vehicles.

Currently, only support for ArduCopter vehicle has been developed in AirSim & ArduPilot.

Note

This has been tested with everything in single Linux machine with Ubuntu 16.04, as well as with AirSim running in Windows 10 & ArduPilot inside WSL (Ubuntu 18.04)

AirSim is an excellent platform for testing and developing systems based on Computer Vision, etc. on simulated vehicles. It’s a very feature-rich simulator with detailed environments and APIs (Python, C++, ROS) for collecting data. See the main Airsim Readme for details and the features available.

A demo of AirSim running with ArduPilot SITL

A list of topics for easier navigation in the page-

  1. Install AirSim
  2. Setup Unreal Environment
  3. Using AirSim with ArduPilot
  4. Launch Copter SITL
  5. Using Lidar
  6. Manual Flying using RC
  7. Multi-Vehicle Simulation
  8. Custom Environment
  9. Using AirSim APIs
  10. Run on Different Machines
  11. Debugging and Development Workflow

Installing AirSim

Binaries are available for AirSim, but since this is a very recent development, you’ll have to build it yourself (on Windows as well as Linux).

Using the precompiled environments won’t work, only environments whose source code is available such as Blocks or any other environments which you download from the Unreal Marketplace will work. Using the precompiled environemnts will be possible once AirSim has it’s next stable release.

Build on Windows

The main page for Windows setup is here, but the instructions have been described below as well. Check the FAQs on the page if there are any problems.

  1. Install Unreal Engine
  2. Build AirSim - Follow the steps on AirSim Setup for Visual Studio packages and clone AirSim.

Run build.cmd from the command line. This will create ready to use plugin bits in the Unreal\Plugins folder that can be dropped into any Unreal project.

Build on Linux

AirSim’s page on Linux Setup is here, please see the FAQs if there are any problems

  1. Build Unreal Engine - After registering with Epic Games, you’ll need to link your Github account with this, please see this page on how to link it.
  2. Build AirSim
  1. Clone the repository

    git clone https://github.com/Microsoft/AirSim.git
    
  2. Build it

    cd AirSim
    ./setup.sh
    ./build.sh
    

Setup Unreal Environemt

Finally, you will need an Unreal project that hosts the environment for your vehicles. AirSim comes with a built-in “Blocks Environment” which you can use, or you can create your own.

Blocks is not a very highly detailed environment to keep the repo size reasonable but it’s useful for various testing all the times and it is the easiest way to get your feet wet in this strange land.

See AirSim’s Blocks Setup page for running the environment.

For using another environment, check the information at the end of the page.

Using AirSim with ArduPilot

Make sure that you have setup ArduPilot SITL, completed the Unreal Environment setup and verified that both are working individually before proceeding.

Note

Go to Edit->Editor Preferences, in the Search box type CPU and ensure that the Use Less CPU when in Background is unchecked.

AirSim’s settings.json file specifies the vehicle and it’s various properties. See the page for the options available.

It’s stored in at the following places- Windows: Documents\AirSim, Linux: ~/Documents/AirSim

The file is in usual JSON format. On the first startup, AirSim would create settings.json file with no settings.

Launch Copter SITL

For using ArduCopter, the settings are as follows-

{
  "SettingsVersion": 1.2,
  "LocalHostIp": "127.0.0.1",
  "LogMessagesVisible": true,
  "SimMode": "Multirotor",
  "OriginGeopoint": {
    "Latitude": -35.363261,
    "Longitude": 149.165230,
    "Altitude": 583
  },
  "Vehicles": {
    "Copter": {
      "VehicleType": "ArduCopter",
      "UseSerial": false,
      "DefaultVehicleState": "Disarmed",
      "UdpIp": "127.0.0.1",
      "UdpPort": 9003,
      "SitlPort": 9002
    }
  }
}

First launch AirSim, after that launch the ArduPilot SITL using

sim_vehicle.py -v ArduCopter -f airsim-copter --add-param-file=libraries/SITL/examples/Airsim/quadX.parm --console --map

Note

Initially, the editor will hang after pressing the Play button if the ArduPilot SITL hasn’t been started (this is due to Lock-Step Scheduling). Run sim_vehicle.py and it should go back to normal.

For closing, first stop the AirSim simulation by pressing the Stop button, then close ArduPilot. If ArduPilot is closed first, then UE hangs and you’ll need to force close it.

You can restart by just pressing the Play button and then start the ArduPilot side, no need to close the Editor completely and then start it again.

Using Lidar

See Lidar Settings for info on Lidar and it’s properties in AirSim.

Current settings.json file for launching ArduCopter with Lidar

{
  "SettingsVersion": 1.2,
  "LocalHostIp": "127.0.0.1",
  "SimMode": "Multirotor",
  "OriginGeopoint": {
    "Latitude": -35.363261,
    "Longitude": 149.165230,
    "Altitude": 583
  },
  "Vehicles": {
    "Copter": {
      "VehicleType": "ArduCopter",
      "UseSerial": false,
      "DefaultVehicleState": "Disarmed",
      "UdpIp": "127.0.0.1",
      "UdpPort": 9003,
      "SitlPort": 9002,
      "AutoCreate": true,
      "Sensors": {
        "Imu": {
          "SensorType": 2,
          "Enabled": true
        },
        "Gps": {
          "SensorType": 3,
          "Enabled": true
        },
        "Lidar1": {
          "SensorType": 6,
          "Enabled": true,
          "NumberOfChannels": 1,
          "PointsPerSecond": 5000,
          "DrawDebugPoints": true,
          "RotationsPerSecond": 10,
          "VerticalFOVUpper": 0,
          "VerticalFOVLower": 0,
          "HorizontalFOVStart": 0,
          "HorizontalFOVEnd": 359,
          "DataFrame": "SensorLocalFrame"
        }
      }
    }
  }
}

Launch Copter with Lidar using

sim_vehicle.py -v ArduCopter -f airsim-copter --add-param-file=libraries/SITL/examples/Airsim/quadX_lidar.parm --console --map
Manual Flying using RC

For flying manually, you need a Remote Control or RC.

Just plug the device in the computer and it should work. See AirSim’s Remote Control page for details on supported devices and FAQs.

Note

This feature hasn’t been tested properly as of now so you might need to modify the Joystick file as mentioned in the page or set some RC parameters, especially if using a different controller.

Multi-Vehicle Simulation

For simulating 2 copters, an example script has been added which will create 2 copter instances and enable Follow mode in one of them.

settings.json for 2 copters

{
  "SettingsVersion": 1.2,
  "LocalHostIp": "127.0.0.1",
  "SimMode": "Multirotor",
  "OriginGeopoint": {
    "Latitude": -35.363261,
    "Longitude": 149.165230,
    "Altitude": 583
  },
  "Vehicles": {
    "Copter1": {
      "VehicleType": "ArduCopter",
      "UseSerial": false,
      "DefaultVehicleState": "Disarmed",
      "UdpIp": "127.0.0.1",
      "UdpPort": 9003,
      "SitlPort": 9002
    },
    "Copter2": {
      "VehicleType": "ArduCopter",
      "UseSerial": false,
      "DefaultVehicleState": "Disarmed",
      "UdpIp": "127.0.0.1",
      "UdpPort": 9013,
      "SitlPort": 9012,
      "X": 0, "Y": 3, "Z": 0
    }
  }
}

Press Play, cd to ardupilot directory then run the script to launch 2 copter instances. You can optionally specify the IP address of the computer with the GCS as the first argument, by default it’ll be 127.0.0.1, meaning everything is on the same computer.

libraries/SITL/examples/Airsim/follow-copter.sh <IP>

To attach MAVProxy -

mavproxy.py --master=127.0.0.1:14550 --source-system 1 --console --map

This will bring up the map but with only a single vehicle, use the vehicle command to switch between controlling the vehicles such as with vehicle 1 & vehicle 2, after which both the vehicles should be appearing on the map

Now, you can have the first vehicle (i,e with SYSID 1) flying in Guided or Auto Mission, and then takeoff the second vehicle and put it in Follow mode, after which the second copter will follow the first one.

For increasing the number of simulated vehicles, just modify the NCOPTERS variable in the script and add the settings for each individual vehicle in the settings.json.

Note

There can be certain problems while working on multi-vehicle simulation due to networking differences between platforms such as Linux, WSL, Cygwin, etc. This Discuss thread could be helpful in such cases.

Note

The difference of 10 between the ports is important since the script is launching the vehicles using the instance option which increases the ports from ArduPilot’s side by 10. For using different ports, modify the script as required following the instructions at the end of the page for specifying the ports.

ROS with Multi-Vehicle Simulation

Using ROS for multi-vehicle tasks is a common usecase and Mavros is used for working with Mavlink-based vehicles. There are some example scripts demonstrating how to use Mavros with multiple vehciles in ArduPilot.

First is the multi_vehicle.sh script which launches multiple ArduCopter binaries with different SYSIDs and ports for each vehicle. Usage is similar to the above script -

libraries/SITL/examples/Airsim/multi_vehicle.sh <IP>

The multi_uav_ros_sitl.launch file demonstrates how to write a launch file controlling multiple vehicles with Mavros. It creates a different namespace for each drone and each drone has a seperate SYSID and ports according to how the script sets the variables. Launching the file -

roslaunch libraries/SITL/examples/Airsim/multi_uav_ros_sitl.launch

Seperate MAVProxy instance can be launched for each drone by connecting to the TCP ports opened by the script for each drone. The UDP ports can’t be used for this if Mavros is already running since Mavros will use the UDP ports.

The multi_vehicle.sh script doesn’t enable the Follow Mode, but if this is also needed and if all the vehicles are to be displayed on the same GCS, then multicast and the Follow parameters as done in the follow-copter.sh script can be added.

Custom Environment

For using another environment on Windows, see AirSim’s custom env setup page.

Linux

As mentioned in the above-linked page, there is no Epic Games Launcher for Linux which means that if you need to use a custom environment, you will need Windows machine to do that.

The steps are the same once you have the Windows machine, after you have downloaded the Unreal project, just copy the project over to your Linux machine.

Follow the steps till after Step 6 where you have edited the .uproject file. After editing the project file, skip Step 7,8 and directly start the Editor by going to UnrealEngine folder and start Unreal by running UnrealEngine/Engine/Binaries/Linux/UE4Editor.

When Unreal Engine prompts for opening or creating project, select Browse and select your custom environment. Afterwards, continue following the Steps from 9 onwards.

Note

When using a custom environment, it might be the case that there are multiple Player Start objects. In such a case, it randomly chooses one and the vehicle can start in the air and fall.

You’ll have to delete the extra Player Start objects and leave one which has to be moved to near the ground. See this excellent video by one of the AirSim developers - Unreal AirSim Setup, specifically at 5:00 where it’s demonstrated how to delete the objects and to move the position.

Using AirSim APIs

AirSim’s APIs document explains the different APIs available and their usage.

Currently, ArduCopter vehicle doesn’t support controlling the drone through the AirSim APIs, however any method of controlling the movement which connects directly to ArduPilot rather than using AirSim’s API work, examples include DroneKit & ROS with Mavros

The Image APIs have been tested to work with Copter, for some ready-to-run sample codes, see the files in PythonClient/multirotor such as opencv_show.py.

A ROS wrapper has also been added. See airsim_ros_pkgs for the ROS API, and airsim_tutorial_pkgs for tutorials.

Note

Not all the APIs have been tested with Copter, if you find things that don’t work or would like to have them supported, please let us know

Run on different machines

  1. Change UdpIp in the settings.json file to the IP address of the machine running ArduPilot
  2. Use -A option to pass the next arguments to the SITL instance, followed by --sim-address to specify Airsim’s IP address

An example-

sim_vehicle.py -v ArduCopter -f airsim-copter --add-param-file=libraries/SITL/examples/Airsim/quadX.parm --console --map -A --sim-address=127.0.0.1
Using different ports

UdpPort denotes the port no. which ArduPilot receives the sensor data on (i.e. the port that Airsim sends the data to)

SitlPort assigns the motor control port on which Airsim receives the rotor control message

  • --sim-port-in should be equal to sensor port i.e. port specified in UdpPort
  • --sim-port-out should be equal to motor control port i.e. port specified in SitlPort

Similar to changing the IP address as mentioned above, use -A to pass the arguments to the SITL instance.

Development Workflow

AirSim’s Development Workflow page explains the recommended setup for developing Airsim on Windows.

For Linux, make code changes in AirLib or Unreal/Plugins folder and then run ./build.sh to rebuild. This step also copies the build output to Blocks sample project. You can then follow the steps to start Unreal Editor and launch the project. When prompted about missing .so files, press Yes to build it again.

Linux Troubleshooting

Windows FAQs

General FAQs

Before reporting any problems, please update the ArduPilot and AirSim installations to the latest master. After updating the local AirSim repository, make sure to run the commands mentioned in the Unreal Environment Setup page, otherwise the updates won’t be reflected in the simulation.