A Solution To Unity's Camera.WorldToScreenPoint Causing UI Elements To Display When Object Is Behind The Camera

A Solution To Unity's Camera.WorldToScreenPoint Causing UI Elements To Display When Object Is Behind The Camera

If you've ever worked with Camera.WorldToScreenPoint to get object positions on the screen from their world space positions, and then displayed some sort of UI element on it, you may have noticed that when your camera faces away from the objects, you still get your UI element displaying on the screen. This is due to Camera.WorldToScreenPoint using an infinite line through the object and through the screen. If you search on Google or the Unity Forums/Answers, you'll get three solutions for this:

  • Use Camera.WorldToScreenPoint's "z" value to determine when the object has passed the screen's plane, and then disable the UI element.
  • Use Vector3.Dot with the camera's transform.forward direction with the direction towards the object to find out if the object has passed the screen's plane and if so, disable the UI element.
  • Use a Rect of (0, 0, Screen.width, Screen.height) to determine if it contains the object's screen point, and disable the UI element.

These solutions may or may not work well depending on the type of UI element you're using. Since I'm using a selection box in my prototype as you can see above, which is drawn by getting the corners of an invisible collider box around the spacecraft, I get a lot of weird issues depending on the angle of the camera and how close it is to the object. Using the solutions above solves one problem but leaves others. After much trial and error, I found a solution after getting a better understanding of how Camera.WorldToScreenPoint gives screen coordinates when the object is behind the camera. I'm posting it here because I haven't seen anyone post anything similar online.

Here is the code:

// This method calculates the screen space rect from the bounds of the unit's Bounds collider.
private Rect GetScreenRectFromBounds(Bounds bounds)
{
  // For less typing and more clarity.
  Vector3 cen = bounds.center;
  Vector3 ext = bounds.extents;

  // There are 8 corners of a rectangle bounding box. Get the screenspace coordinate of each corner. We are using the
  // array member variable to save allocations when you create a new array each frame.
  screenBoundsExtents[0] = mainCamera.WorldToScreenPoint(new Vector3(cen.x - ext.x, cen.y - ext.y, cen.z - ext.z));
  screenBoundsExtents[1] = mainCamera.WorldToScreenPoint(new Vector3(cen.x + ext.x, cen.y - ext.y, cen.z - ext.z));
  screenBoundsExtents[2] = mainCamera.WorldToScreenPoint(new Vector3(cen.x - ext.x, cen.y - ext.y, cen.z + ext.z));
  screenBoundsExtents[3] = mainCamera.WorldToScreenPoint(new Vector3(cen.x + ext.x, cen.y - ext.y, cen.z + ext.z));
  screenBoundsExtents[4] = mainCamera.WorldToScreenPoint(new Vector3(cen.x - ext.x, cen.y + ext.y, cen.z - ext.z));
  screenBoundsExtents[5] = mainCamera.WorldToScreenPoint(new Vector3(cen.x + ext.x, cen.y + ext.y, cen.z - ext.z));
  screenBoundsExtents[6] = mainCamera.WorldToScreenPoint(new Vector3(cen.x - ext.x, cen.y + ext.y, cen.z + ext.z));
  screenBoundsExtents[7] = mainCamera.WorldToScreenPoint(new Vector3(cen.x + ext.x, cen.y + ext.y, cen.z + ext.z));

  // Set these variables for a safe margin distance around the unscaled canvas which we can set things to. These can be located outside of this method, but are shown here for clarity. If left here, it can handle dynamically changing the resolution.
  int margin = 20;                                // Indicates the size of the margin outside of the canvas on all sides we are willing to let the indicators stretch to.
  int minimum = -margin;                          // Beyond the left and bottom of the screen.
  int maximumWidth = Screen.width + margin;       // Beyond the right side of the screen.
  int maximumHeight = Screen.height + margin;     // Beyond the top of the screen.


  // The following section is used instead of Vector2.Min and Max, as they cause drawcalls.

  // These variables will hold the screenspace bounds of the object. We're initializing to the margin to the left and bottom of the canvas.
  float xMin = minimum;
  float xMax = minimum;
  float yMin = minimum;
  float yMax = minimum;

  // Now that we've done that, we find the first screenpoint of one of the bounds that is in front of the camera plane, and setting that as the first to be compared against.
  for (int i = 0; i < screenBoundsExtents.Length; i++)
  {
    if (screenBoundsExtents[i].z > 0)
    {
      xMin = screenBoundsExtents[i].x;
      xMax = screenBoundsExtents[i].x;
      yMin = screenBoundsExtents[i].y;
      yMax = screenBoundsExtents[i].y;
      // Break out of the loop as we don't need to loop further.
      break;
    }
  }

  // To save repeated calculations.
  float widthMiddle = Screen.width * 0.5f;
  float heightMiddle = Screen.height * 0.5f;

  // Now we go through each element of the array again to do various things.
  for (int i = 0; i < screenBoundsExtents.Length; i++)
  {
    // If this particular point is behind the camera.
    if (screenBoundsExtents[i].z <= 0)
    {
      // The following comparisons are the heart of this solution. Due to the way Camera.WorldToScreenPoint works,
      // any point behind the camera gets flipped to the opposite sides.
      // Therefore, if the point is to the left of the middle of the screen width, we put it on the right outside of the canvas.
      // If the height is less than the middle of the screen height, then we put it above the canvas, etc.
      // This allows us to have the indicator appear in the right places as it should when we're very close to or passing through an object.

      // Width checks of the point.
      if (screenBoundsExtents[i].x <= widthMiddle)
        screenBoundsExtents[i].x = maximumWidth;
      else if (screenBoundsExtents[i].x > widthMiddle)
        screenBoundsExtents[i].x = minimum;
      // Height checks of the point.
      if (screenBoundsExtents[i].y <= heightMiddle)
        screenBoundsExtents[i].y = maximumHeight;
      else if (screenBoundsExtents[i].y > heightMiddle)
        screenBoundsExtents[i].y = minimum;
    }

    // Every point will be checked now that they're put in the appropriate place in screen space, even if they're behind the camera.
    // Find the values which comprise the extents of the bounds in screen space by saving it to the previously declared variables.
    if (screenBoundsExtents[i].x < xMin)
      xMin = screenBoundsExtents[i].x;

    else if (screenBoundsExtents[i].x > xMax)
      xMax = screenBoundsExtents[i].x;

    if (screenBoundsExtents[i].y < yMin)
      yMin = screenBoundsExtents[i].y;

    else if (screenBoundsExtents[i].y > yMax)
      yMax = screenBoundsExtents[i].y;
  }

  // Clamp all the values now, so we don't crash the editor with large screenspace coordinates.
  // This is done here rather than an if (screenBoundsExtents[i].z > 0 block because if all points are in front of the camera,
  // there is a possibility of still crashing the editor.
  xMin = Mathf.Clamp(xMin, minimum, maximumWidth);
  xMax = Mathf.Clamp(xMax, minimum, maximumWidth);
  yMin = Mathf.Clamp(yMin, minimum, maximumHeight);
  yMax = Mathf.Clamp(yMax, minimum, maximumHeight);

  // Now we have the lowest left point and the upper right point of the rect. Calculate the
  // width and height of the rect by subtracting the max minus the min values and return it as a rect.
  return new Rect(xMin, yMin, xMax - xMin, yMax - yMin);
}

*Disclaimer* I'm not an advanced programmer, but still wanted to share what I found to help others stuck with the same problem. There may be a better way of doing this, but the heart of the solution is in the block:

if (screenBoundsExtents[i].z <= 0)

You can see the reasoning behind the solution in the code comments.

To observe how this works, have another scene view open while the Unity editor is in Play mode. Make it 2D and display the UI Canvas within the scene view. If you're using a selection or highlight indicator of some sort, you can see how it stretches and where it's located, especially when the camera is looking away from the object. All the best!

Comments

The comments shown below are from visitors when I was using Squarespace for website hosting, but I've switched to cheaper hosting with a different CMS and currently have registering/commenting disabled. I've copy/pasted all past comments in case they may be useful for someone.

Cheeze 2021

Thank you for the blog post! It seems that the GitHub link does not appear to be working. Is the example still available?

Sean Kumar 2021

Hey, thanks for bringing this to my attention. Apparently Github deleted my gist without telling me. Thankfully I found the code using the Internet Archive Wayback Machine and posted it directly in the blog post above. Hope you find it useful!

alealeonardkrea 2020

Thank you for this! I was just in the middle of creating a target indicator and ran into the issue with the camera going behind the canvas. I was scratching my head on how I was going to fix it and came across your entry.

Sean Kumar 2020

It's great to see that this was useful! It's been a while so I don't even remember what I did.