Raising the temperature

I flew Phoebe a few times yesterday (with blu-tack), and she consistently drifted several meters during a 10s flight – worse than ever.

This morning I brought her indoors just to confirm this massive deterioration in flight stability was temperature related.  Sure enough, indoors she just hovered perfectly:

Checking the temperature from Andy Baker on Vimeo.

Another temperature test

I ran a quick passive exploratory test tonight, again temperature related with the blu-tack installed.  The aim was to understand how much the IMU temperature drift is related to being worked hard, and how much is related to the wind from the props.

I ran a test indoors with no prop power – here’s what I got.  Look at the resolution of the temperature: over an 11 second flight, the temperature range spans 35.79 – 35.93°C, or 0.14°C peak-to-peak which is tiny.  I think I can safely say it’s solely environmental changes (i.e. the breeze front the props) which cause IMU temperature- and therefore sensor drift.

Passive flight temperature change

Passive flight temperature change

I think I need to add some more blu-tack to the underside of the IMU PCB to give it the same breeze resistance the top now has.

Phoebe’s nemesis returns

I took Phoebe out this morning for get video proof of her new low drift 12 second flights.  But she drifted – up to 4 meters.  The code was untouched so what had changed?  All I could think of was her nemesis: the sensor drift against temperature.

Phoebe self-calibrates for this prior to each flight while she’s sitting on the ground with no props moving: the butterworth is set to let through low frequency sensor drift.  What was special about this morning is that the ambient temperature was low – about 10°C; during the 20s calibration prior to flight, the IMU would have warmed up to an extent the butterworth could handle.  My speculation was that as she leapt off the ground immediately after calibration, the draft from the props acted like cooling fans, and because of the the cool ambient temperature, they worked very well to the extent the sensor readings started drifting faster than the dynamic calibration of the butterworth could keep up with.

This was very very speculative, but I could test the theory.  By sticking a lump of blu-tack on the IMU, it was thermally insulated and protected from the wind; sure enough, the drift was much reduced.  However, blu-tak is opposite of the cooling fan problem.  It’s thermal insulation means the IMU could now warm up during flight as it couldn’t dissipate heat.

What’s really needed is a box to block the breeze from the props while allowing the heat the IMU generates to dissipate gradually allowing the Butterworth to keep up with the temperature drift.  I’ll do some more tests with the blu-tack in the cool of tomorrow morning, but I suspect my speculation might actually be fact.

Integration errors?

Integration Errors?

Integration Errors?

I took Phoebe out for a quick flight this morning on the drive – Excel doesn’t have a decent colour to represent gravel, so I’ve stuck with the grass colour scheme. I added some dots on the graph spaced by about 10ms so it’s possible to see when she thinks she’s travelling slowly (high density of dots) or fast (low density of dots).

The graph shows good stability until towards the end when she starts moving backwards at higher speed – note the scales – the drift left is tiny compared to the backward movement.

From my point of view, she was pretty stable for most of the time, but then headed forwards to a brick wall towards the end of the flight.

She flew with every piece of refinement code enabled – specifically 0g offset calibration is back, and I calibrated her last night.

The fact the drift increases significantly towards the end of the flight suggests cumulative integration errors.  There are at least two possible causes:

  • sensor drift
  • missed samples

I wonder how much the missed samples plays a role?  Sensors churn out samples at 1kHz, and motion processing happens at about 100Hz based upon 10 averaged samples.  The main processing loop actually happens about 950Hz – the motion processing is occasionally causing the code to miss about 5% of the samples.  It’s certainly worth investigating further, although she’s already overclocked to 900MHz, and increasing that previously caused errors.  I don’t think there’s much more I can do with the hardware interrupt processing, and I have her process running at a higher priority (lower nice setting) than the rest of the system.

On with the thinking cap!

Phoebe’s Playtime in the Park

I took Phoebe down to the local play park to try some extended flights without the risk of her crashing into solid objects – I’m on my last set of props and I really can’t afford to keep replacing them several times a month.

Anyway, net of the flights was she’s good for a few seconds as shown in yesterday’s video, but extending the hover time to 10s from yesterday’s 4 shown accelerating drift.

Best guess is longer term accuracy of angles using just integrated gyro reading, and I need to reintroduce the complementary filter to merge in the angles from the accelerometer to provide longer term stability.

Oh well, it was worth a try…

Sensor vs Reality

Sensor vs Reality

Phoebe thinks she’s drifting left according to the sensor data shown in the graph, but in real life, I watched her drift right towards the house in compensation for what she perceived as leftward drift. All I can do about that is 0g offset calibration, and all previous attempts have failed. It’s not just Phoebe who can see a brick wall heading her way.

Test flight with new angles

I took Phoebe out to try the new angles code in a small window between torrential rain and winds gusting to over 20mph.  The weather made it hard to draw any solid conclusions, but certainly her behaviour was no worse than prior to the new code.  She took off and hovered at a fixed height, but with some lateral drift that looked like she was correcting, albeit slowly.  So it’s definitely worth further testing on a day with better weather conditions.

Next step will be to collect flight diagnostics to see whether what I see happening matches what her sensors believe is happening; this’ll give real evidence whether the new way of calculating angles is better without the complementary filter for removing accelerometer noise, and whether the use of the low-noise gyro for angle increments results in better readings for lateral and vertical drift matching what’s happening in the real world.

Looks like Thursday is predicted to have the conditions I’m looking for.  Fingers crossed.

A Valentines Kiss

Keep it simple, stupid!

I have a nasty feeling that the months and months of hard work I put into calibration of the sensors against temperature have been a complete waste of time; I’ve always wondered how off-the-shelf quadcopters work without this level of calibration and now I think I know.

The MPU-9250 and MPU-6050 product specs defined that the drift against temperature in 0g conditions is ±1.5mg/ºC – that’s 1.5 cm/s2 drift as the temperature changes.  That means that as long as the temperature remains vaguely constant during a flight, then the value effectively doesn’t change.  Add to that the Butterworth filter to dynamically extract gravity from accelerometer readings, and rotation matrices so that Butterworth is applied in the earth frame (i.e. 0, 0, 1g), then the X, Y and Z accelerometer readings should be stable in hover regardless of ambient temperature as long as ambient varies more slowly than it takes for the Butterworth to track any sensor drift.  Any offset and gains from (lack of) calibration are neutralized because of the subtraction of gravity from acceleration, both of which share the same offset / gain errors

This realization was all triggered by the fact the MPU-9250 specs don’t give a way to calculated absolute temperature, and initial testing shows widely different values depending on the ambient temperature the chip was powered up in e.g. powering up the IMU in a beer fridge and read the temperature sensor gives a hugely different value to powering it up in the ‘engineering lab’ and then putting it into the beer fridge and reading the temperature.  This meant I couldn’t do sensor calibration at a fixed temperature (40ºC for Phoebe, Chloe and Zoe).

So all I need to do prior to a flight is allow enough time for the Butterworth to settle while on the ground.  Oh, and delete reams of code, and throw away all the calibration equipment.

But I’m not too disconsolate – I couldn’t have got to this stage without Butterworth filtering for gravity, and I couldn’t have got there without rotation matrices, and I couldn’t have go there without precalibrated gravity drifting within flight at constant temperature and I couldn’t have got there without stable temperature calibration and I couldn’t have got there without … – well you get the idea.

For anyone reading the BYOAQ-BAT thread, don’t worry, that hasn’t got to the calibration stage yet, so everything so far has still necessary.  But once more, BYOAQ-BAT is now stalled pending testing of the above.

Accelerometer drift proof and solutions

These are readings direct from the 3 accelerometers with virtually zero code tinkering – specifically no subtraction of gravity; the only code modifying the sensor data values is fixed calibration values, fixed unit conversion and averaging. None of these three should cause the decline in the Z-axis (yellow) line against the right-hand axis.

Z-axis drift

Z-axis drift

So I need to do something about this, in particular the Z axis which is the cause of my ever climbing flights.  3 approaches:

  1. give in and add an altimeter – but I’m waiting for the next revision of the DroTek board that has the pin spacing I needs – however this wouldn’t be ideal for indoor flights due to air pressure changes as people open doors etc
  2. give in and add ultrasonic range finders – again not perfect for a variety of reasons – firstly it’s I2C is only 100kHz rather than the 400kHz for the MPU6050 so could slow things down without some very careful coding;  secondly it’s tracking distance to ground where as the accelerometer is tracking from takeoff point – so in a lecture theatre, takeoff from a table would give horizontal flight from the accelerometer, but a rapid descent as the quad passes over the edge of the table, also possible causing two of the arms to clip the table; third and finally, I’m not sure how well a URF will perform on grassy rather than hard surfaces.
  3. filter the accelerometer output to smooth the acceleration (the spike at one second in the graph above) but trace the steady drift downwards; simplest would be yet another complementary filter, merging in fractions of latest accelerometer readings into the initial value in the earth axis; better though would probably be a Kalman filter where the prediction it provides may well be able to remove the spikes completely – it’s also easier as there’s just a single source of the data being filtered, unlike the current complementary filters which are used to merge angles from gyro and accelerometer.

So option 3 it is, though as always, I’ll chicken out on the Kalman unless the complementary doesn’t work well!

Oh what now?

I managed to squeeze in another flight today, and got diagnostics. The flight was only 3 seconds before I had to kill it – she then flipped over and landed upside down on the roof before sliding down (phew) and landing on the grass.  No damage done to the house or Phoebe.

During the first second or so, the flight plan was zero movement in all three axes while the props got up to speed. For the next two seconds, Phoebe rose, but she rose to over 3m in those two seconds instead of the target of one meter.  She also drifted gently to the right and possibly backwards (but that’s harder to see from behind).

This graph shows accelerometer readings over the course of the flight, with gravity (a fixed reading taken during the flight warm-up period and rotated to the same orientation) subtracted:

Acceleration Drift

Acceleration Drift

I’ve added linear trend lines to the graph which, at least over these 3 seconds, do fit with a linear drift in the accelerometer readings.  Note that the drift in readings seems independent of whether she’s stationary on the ground for the first second, or climbing for the remaining flight time.

So what could be the cause?  For once, it isn’t my code – at this point no motion processing has happened – this is raw data read from the sensors.  It also can’t be calibration as the trend lines would be horizontal with a fixed error.

My first suspect for the drift was temperature, but this graph of sensor temperature over the course of the flight shows that’s unlikely:

Flight temperature

Flight temperature

Could this be dlpf filtering out important stuff or including unnecessary stuff?  It was set to 3 – 44Hz and 4.9ms lag.

I have 2 plans of attack here:

  1. try upping the dlpf to 2 – 94Hz and 3ms lag and down to 4 – 21Hz and 8.5ms lag and see what happens.
  2. try physical filtering by isolating the sensors from the prop noise.

Option 2 is Zoë – the genetically engineered clone of Phoebe’s software and electronics with Chloë’s hardware, with a special dash of silicone dampers to isolate the Raspberry Pi and Beret lower platform from the noise from the props / motors transmitted along the arms to the top platform.  The remaining parts are arriving tomorrow.

I do hope this works because otherwise this project has just hit the biggest metaphoric brick-wall possible – think of the great wall of China, but built of granite.