I fixed it. Problem was my input and output generalized poorly. It couldn't learn from one corner and then directly apply that to the next. I wanted it to learn how to interpret the input and output data itself, but that was asking too much for how simple the net is.
All I needed to do was rotate the velocity output to match the car and it loops the track with ease.
All I needed to do was rotate the velocity output to match the car and it loops the track with ease.