Camera API Discussion

Hello Developers,

I write this article to present you a draft of the Camera API that we are developing (SU5, SU6 no ETA yet). We have read the numerous requests about this topic along those years on DevSupport and tried to answer as many wishes as possible. Before publishing it, we want to be sure that we are not missing any important parts, so feel free to share with us any thoughts about these headers and any future needs you may have (related to this topic) that could help us take this API in the right direction. As always, if you can detail your requests/thoughts with examples and or information, it would help us a lot. The more explicit and concrete you are, the more precisely we can answer.

Flow

A dedicated camera is reserved to addons. This camera can be acquired by SimConnect clients and/or Wasm modules. Once a client possesses the camera, no other addon can steal it from him. That being said, it is not because a client has acquired the camera that it has all rights on the sim’s cameras :

  • The Sim can temporary take back control over the cameras during specific part of the flow (menu, pause screen, RTC…). Passed those situations, the control of cameras will be given back to the addon which had acquired the dedicated camera.

  • The User has all rights over the cameras. Through the camera ingame panel (and maybe others places), he can revoke the acquisition of the addon camera and/or forbid its future acquisition.

All those information will be sent to all clients which have subscribed to this event, even if they haven’t acquired the camera.

Feature

Usable by everyone

CameraAcquire

Try to acquire the camera. Depending on the technology used, the call will look like

SimConnect

HRESULT SimConnect_CameraAcquire(HANDLE hSimConnect);

Send an exception if an internal error occurs. Send a message which contains the same information described in CameraGetStatus (see below) otherwise.

Wasm

bool fsCameraAcquire();

Returns true if the camera has been acquired. Returns false otherwise.

CameraGet

Get the current settings of the camera which are composed of the following data:

  • Position (X, Y, Z). The units of those values will depend of the selected referential (see below)

  • Pbh

  • FOV (double)

  • Referential. One of the following values:

    • Aircraft The position is an offset from the aircraft’s datum (expressed in meter)

    • Eyepoint The position is an offset from the aircraft’s eyepoint (expressed in meter)

    • World The position is a LLA

SimConnect

HRESULT SimConnect_CameraGet(HANDLE hSimConnect, DWORD referential);

Send an exception if an internal error occurs. Send a SIMCONNECT_RECV_CAMERA_STATUS (see below) otherwise.

Wasm

bool fsCameraGet(int referential, FsCameraData* pOut);

Fill the structure given in parameter with the data detailled above. Returns false if something went wrong. Returns true otherwise.

Note : It is possible to use a different referential from the one used by the camera. In this case, the position will be converted to the corresponding referential.

CameraGetStatus

Get the status of the shared camera, which is composed of the following data:

  • AcquiredState : One of the following values:

    • NOT_ACQUIRED

    • ACQUIRED

    • ACQUIRED_BY_OTHER

    • USER_DISABLED

  • Sim Control State : a boolean

SimConnect

HRESULT SimConnect_CameraGetStatus(HANDLE hSimConnect);

Send an exception if an internal error occurs. Send the same message as the one detailled in above

Wasm

bool fsCameraGetStatus(FsAcquiredState* outAcquiredState, bool* bSimControlled);

Returns false if an internal error occurs. Returns true otherwise

SubscribeToCameraStatusUpdate

Subscribe to this event which will send a message to the client each time the camera status is updated, to avoid using CameraGetStatus each frame

SimConnect

HRESULT SimConnect_SubscribeToCameraStatusUpdate(HANDLE hSimConnect);

Send an exception if an internal error occurs. Send the same message as the one detailled in CameraGetStatus

Wasm

bool fsCameraSubscribeToStatusUpdate(fsCameraWasmCallback callback, void* context);

Callback to call on event with the given context. Return false if an internal error occurs. Returns true otherwise.

UnsubscribeToCameraStatusUpdate

Unregister all callbacks registered for this event.

SimConnect

HRESULT SimConnect_UnsubscribeToCameraStatusUpdate(HANDLE hSimConnect);

Send an exception if an internal error occurs.

Wasm

bool fsCameraUnsubscribeToStatusUpdate();

Return false if an internal error occurs. Returns true otherwise.

EnumerateCameraDefinitions

Enumerate all camera definitions linked to the selected aircraft

SimConnect

HRESULT SimConnect_EnumerateCameraDefinitions(HANDLE hSimConnect);

Sends messages containing the list of all camera definitions name.

Send an exception if an internal error occurs.

Wasm

bool fsCameraEnumerateCameraDefinitions(char** outArrCameraDefinition, int* outArrSize);

Return false if an internal error occurs. Returns true otherwise.

The returned list (char**) must be freed by the user.

Usable if you have acquire the camera

CameraRelease

Release the camera previously acquired. The camera will be available for others to acquire. It is possible to specify the camera definition name towards which the camera switches. By not specify it, it will switch the one set before acquired it.

SimConnect

HRESULT SimConnect_CameraRelease(HANDLE hSimConnect, const char* cameraDefinition);

Send an exception if an internal error occurs or if the client is not the owner of the camera.

Wasm

bool fsCameraRelease(const char* cameraDefinition);

Returns false if an internal error occurs or if the client is not the owner of the camera. Returns true otherwise

CameraSet

Set the settings of the camera which are composed of the following data:

  • Position (X, Y, Z). The units of those values will depend of the selected referential (see below)

  • Pbh

  • FOV (double)

  • Referential. One of the following values:

    • Aircraft The position is an offset from the aircraft’s datum (expressed in meter)

    • Eyepoint The position is an offset from the aircraft’s eyepoint (expressed in meter)

    • World The position is a LLA

An additionnal mask must be sent to determine which data is set. This mask is a composition of the following value:

  • POSITION

  • ROTATION

  • FOV

  • REFERENTIAL

SimConnect

HRESULT SimConnect_CameraSet(HANDLE hSimConnect, SIMCONNECT_DATA_CAMERA CameraData, DWORD DataMask);

Send an exception if an internal error occurs or if the client is not the owner of the camera.

Wasm

bool fsCameraSet(FsCameraData* cameraData, int dataMask);

Returns false if an internal error occurs or if the client is not the owner of the camera. Returns true otherwise

CameraEnableFlag

Enable camera specific features. Here is the list of possible values:

  • CAMERA_FLAG_INTERACTION : Are interactions possible using this camera

  • CAMERA_FLAG_ABOVE_GROUND : Name to be determined : Always set the camera altitude above ground and/or buildings

SimConnect

HRESULT SimConnect_CameraEnableFlag(HANDLE hSimConnect, DWORD flag);

Send an exception if an internal error occurs or if the client is not the owner of the camera.

Wasm

bool fsCameraEnableFlag(int flag);

Returns false if an internal error occurs or if the client is not the owner of the camera. Returns true otherwise

CameraDisableFlag

Disable camera specific features. Here is the list of possible values:

  • CAMERA_FLAG_INTERACTION

  • CAMERA_FLAG_ABOVE_GROUND

SimConnect

HRESULT SimConnect_CameraDisableFlag(HANDLE hSimConnect, DWORD flag);

Send an exception if an internal error occurs or if the client is not the owner of the camera.

Wasm

bool fsCameraDisableFlag(int flag);

Returns false if an internal error occurs or if the client is not the owner of the camera. Returns true otherwise

Q&A

What about the previous 6DOF camera?

The 6DOF camera usable through SimConnect is still usable in the same way it was. However, it will now use the same system as the Camera API (using the old 6DOF call will be the same as using the acquire call and set position/rotation in EYEPOINT mode). This implies, that if another addon uses the Camera API and has already acquired the camera, 6DOF call will have no effect.

Wishing you a happy end of the year and all the best for next year!
Best Regards
Maxime & Thibault / Asobo

12 Likes

This is Exciting!

1 Like

Hi Maxime,

Thank you very much for opening the conversation about this API and asking our input on this. We really want this API to be useful and asking for feedback is the best way to ensure this.

Here are a few questions I have this morning, I might post more eventually as they come to mind!

Frame Sync

Camera operations are a very time-sensitive thing. Back in FSX when we had the 6DOF function, the only way to get a smooth camera motion was to spam SimConnect at 120Hz with the hope that the frame got rendered with the latest data available. Even in the best scenarios, this would lead to a few frames of latency and an imperfect frame synchronisation. We had to resort to memory injection to bypass this latency.

Q: What is being done in WASM or through SimConnect to ensure our camera code can access the latest aircraft data (and AI traffic data for tracking shots) after it was processed for the next frame but right before the camera positons are sent to the GPU? The current standalone update in WASM modules happens after the camera is sent to the GPU and would always lead to data being processed 1 frame late.

TrackIR, Tobii & VR

Those are additive cameras currently and while it would make sense for it to stay this way with the camera API, having the ability to set a multiplier on those additives would be incredibly useful. That way we can transition them in/out smoothly when we need to based on what we are trying to do with the camera.

Q: Will those offsets be added on top of the position injected by the camera?

Node visibility

The current system has camera definitions with the ability to show/hide nodes when they are in the way of the camera (for example looking at instruments behind a yoke, the yoke can be hidden in some views).

Q: Are there any way we would trigger those through WASM/SimConnect based on a node name we specify?

Control Assignment conflicts

On a fully custom camera system, we expect people to press shortcuts and access their configured views. This creates an issue if the user has the same control assignments configured both in the sim and in the custom camera system. How will this system handle this?

Q: If the user presses a button to go to Drone mode in our camera system but it’s also assigned in the sim, will the sim kick us out in favor of the default drone camera?

Career and Missions

We appreciate the incredible amount of work and progress that has been done on the career and mission system in the sim. We want people to embrace those and use the systems as they please. Currently, it’s not possible to have a toolbar mod active in most career and missions.

Q: If a custom camera system is developed using this API, Will there be a change to allow 3rd party in-game panels while in missions, careers and activities?

2 Likes
  • SimObject The position is an offset from the simobject’s datum (expressed in meter)

where simobject is Tower, AI aircraft, etc.

Also: Target / LookAt vector
• Instead of only orientation (PBH), some APIs allow specifying a “look-at” point in world coordinates.
• This can simplify camera control when you want to track a specific object.

  • That will make the camera setting so much more usable
3 Likes

Flashlight

This one might not be obvious but, users want to use their flashlight and currently the flashlight only works in CAMERA STATE == 2.

Q: Will we be able to enable the flashlight in all camera modes with new API touches?

1 Like

I think I already know the answer to this, but is there any possibility this will ever be back-ported to 2020?

2 Likes

Hello Maxime, thanks so much for tackling this and reaching out for our feedback, I’m excited! :hugs:

Many requests for the Camera API were due to

Multi Monitor (/ Multi Render Window) Support

Most commonly requested use cases for this are

  • Views for home cockpit builds: Being able to adjust the current side view render windows more flexibly than by just an angle difference from the main view (by position difference etc.); and being able to decouple the movement of the side views from the main view (view coupling doesn’t make sense for window views in home cockpit builds)
  • Taxi cam
  • Showing a set of instruments etc. on a separate monitor
  • Displaying a passenger window view / outside view simultaneously to the main view
  • Shared cockpit setups (being able to control a captain’s seat perspective and a FO seat perspective separately and simultaneously)
  • and more

Unfortunately the way I understand within your current draft of the API none of this is realizable, since…

A dedicated camera is reserved to addons

…there are only plans for being able to control one dedicated camera that works within the main render window, correct?

These would be my propositions:

Multiple dedicated cameras

instead of just one, with the ability to

  • create (and destroy) those dedicated cameras, control them via an ID that you received upon creation
  • assign these dedicated cameras to render windows

and / or

Restore functionality of (multi-) camera access by name (like previously in FSX / P3D)

with the a ability to

  • select which of the pre-defined cameras is shown in a specific render window, this was previously done by selecting a camera via name from a dropdown menu. (This could of course be complemented by API functions to assign named cameras to render views.)
  • control these named cameras (simultaneously / independently), this was previously done via SimConnect_CameraSetRelative6DofByName

That would really be a blast. Happy holidays and a happy new year you guys!! :blush:

1 Like

I would like to know more about how it works when multiple add-ons will try to claim the custom camera: which one will get it ? By loading order ?

If the user has full authority about cameras, this means if we have 2+ add-on all requiring camera access, if the user doesn’t do anything to sort the situation, other add-ons that didn’t get the camera initially, will never get the chance to use it ?

2 Likes

We are currently working on an API for ChasePlane so other developers can interact with our Camera system in instances like this. Please reach out if a native API-level solution cannot be offered.

2 Likes

I have to ask: Why are we reinventing the wheel here? Prepar3D already has a full-featured camera API for SimConnect.

See /SDKv4/sdk/simconnect_api/references/simconnect_references.html#ViewFunctions at their site.

Also, I am not clear if what is being suggested will make any difference to earlier issues I raised with the 6DOF API (see “simconnect-camerasetrelative6dof-camera-is-fixed-and-cannot-be-panned-zoomed” and “simconnect-camerasetrelative6dof-pitch-is-opposite-fsx”)

2 Likes

I would like to highlight one thing fm a very long-time simmer point of view, A very important thing mentioned by @runshotgun already. Missions and career. That’s the sim mode urgently required to be operational with camera API products. And the biggest change made in sim since ever before. So I suppose operating in-game panels in these modes is the major keypoint to be worked out (at the moment we do have to use both the in-sim system and the CP which sorry to say makes it completely useless to buy any kind of addons). SU5 means june, SU6 means next december. So in that case these products are unplayable for the next year or so. As the only mode which is actually working is the Free Flight mode.

1 Like

MSFS started from the FSX SDK, not P3D. And even then, the engine is completely different from what it was. We need an API that integrates with the current system and is flexible enough for things to be built on top of it.

3 Likes

I understand, but there is SimConnect prior art (does it matter if it was invented by LM or MS? Not in my opinion). Inventing something new causes divergence and it’s a pain point for us developers.

1 Like

It does, because MS/Asobo doesn’t own what Lockheed Martin did.

And even if they did, why stick with something that is very limited when they can make one that actually does work for today’s needs from the development community?

I beg to disagree w.r.t to the API: Google LLC v. Oracle America, Inc. - Wikipedia.

If my understanding is correct, the proposal by Asobo falls far short of what is possible using the LM API.

So if I understand this correctly, Asobo could just go and steal LM’s work, put it in their engine and then spend months refactoring the rendering engine to support it, instead of spending a few weeks making one that works for the current needs?

No, that is incorrect. Asobo would implement the API, not steal any code.

I think our difference in opinion boils down to: I don’t want to have to support 2 completely different camera APIs (one for P3D and one for MSFS 2024), when there exists prior art for SimConnect. You don’t care about that and just want a camera API.

In either case, one can only hope that whatever is produced actually works and does what everyone needs!

Finally a system to manage camera api . I think this will bring very good things

Thanks @Arzop

1 Like

Camera API is finally in the scope. :blush:

Two quick questions about the new Camera API:

  1. Can the addon‑controlled camera use effects (e.g., drone focus, night vision, etc.)?
  2. What is the supported FOV range? I would like to have 5° to 150°if possible.

Thanks for considering.

2 Likes