It seems this is the case, because C# does not allow bit shifting operations on these types, which is used to turn these types into 4 or 8 bytes.
I ended up using this code to convert these types without using the builtin BitConverter (which performs a malloc!)
float[] FLOAT = new float[1]; public float ReadFloat () { Buffer.BlockCopy(buffer, (int)readStream.Position, FLOAT, 0, 4); readStream.Position += 4; return FLOAT[0]; } public void Write (float value) { FLOAT[0] = value; Buffer.BlockCopy(FLOAT, 0, buffer, (int)writeStream.Position, 4); writeStream.Position += 4; }