Appendicitis: unexpected problems

These appendices are swelling beyond what’s healthy, so I’m going to stop now with a mix of good and bad news.

She’s back on her feet again, although since I took the photo I’ve been doing some more drift tuning during which she smashed into the house, breaking her wrist.  Luckily I have some spare red arms left from the original DJI F450 kit, so now her front left arm is red instead of white.

Rebuilt Phoebe

Rebuilt Phoebe

I had hoped to also present here a video of a successful drift free flight plus code.  However, as usual, things haven’t gone how I’d hoped.

I wanted the outer PID to control horizontal speed so that the input from the radio control could use desired horizontal speed as the target for the PID.  But it’s been difficult to tune.  The primary gain turns out to be the d gain, with the p gain trying to manage the long term speed – but it’s not working; I think I’m going to have to resort to using the horizontal acceleration which will provide better variety.  Which means a change in code and a bit of a rewrite – won’t take long but not desirable.

I’ll probably test the horizontal speed solution a few more times to improve the tuning, but I do have a bad feeling it’s simply not going to play ball.

Appendix 3: We can rebuild her, we have the technology

I broke Phoebe’s legs in so many ways on Wednesday that rummaging through the wreckage today I found I had enough fixable pieces to put together a leg / ankle / foot combo suitable for flying again. And the weather today is once more on my side.

First test was a sanity check on the autonomous take-off again; checking the stats yesterday showed a bug in the PID gains plus a need for some tuning that hopefully would allow for safer (read less destructive) landings.

That went fine, so onwards (and upwards) with the drift tuning. This time, I had the gentle head-wind I needed for testing, and she held her ground this time. Sadly, landing broke her repaired leg again to the extent it’s now irreparable – at least the new ones are on their way from Poland.

I had hoped at this point to declare now that PHASE 1 OF PHOEBE’S DEVELOPMENT IS COMPLETE!  And after a fashion it is, but…

The test today did still show some drift, but not due to wind, but instead due to non-horizontal take-off platform (I’d been sloppy setting it up).  If she takes off at a slight lean, she’s bound to drift because she’s not pointing straight up.  I think the solution here is for me to split the sensor calibration; the gyros can be calibrated on any stable surface since they just measure the speed Phoebe is pitching, rolling and yawing – if she’s sat still, they can all be safely calibrated to read 0.  But the accelerometer is measuring force, including gravity; it needs to be calibrated so that the X and Y axes read 0g on as perfect a horizontal surface as possible; that then ensures that  when in flight with the X and Y axes reading 0g force, that absolutely does represent no horizontal drift.  Any tilt in the calibration surface will cause the X and Y axes to read 0g when in fact there is a tilt and therefore drift.

The calibration values are already written to / read from file anyway, so the changes are simple – split them into two, and add a specific command line option to only do accelerometer calibration on as near perfect a surface as possible.

P.S.  Once she has a compass / magnetometer then this will cease to be an issue. – unfortunately that might be on hold for a while until Invensense ship the MPU9250 and Flyduino produce the corresponding breakout for it.

P.P.S.  Actually, I don’t think I need to split the calibration code, at least not for initial testing; I can do the calibration of both gyro and accelerometer indoors – that saves the results to file anyway, and then just carry her outside to test the theory.

Actually, that was a useful crash…

In yesterday’s set of flights, I’d noticed something worrying a few seconds before the railing impact that I couldn’t explain, and as I was browsing the stats, there is was again: at around 5 seconds there’s a spike in speed, height and power that I simply couldn’t explain; the accelerometer output showed no unusual spikes which could explain why.

Railing crash stats

Railing crash stats

The problem at around 5s was that instead of the code loop taking 10ms, it took about 0.75s; due to this, the next integral output for the vertical speed PID grew by a factor of 75, causing the spike. I did wonder whether it was related to using the RaspiCam, but still couldn’t understand how.

So I posted on the Raspberry Pi Camera forum asking for help / suggestions / solutions. That produced a wealth of explanation (speculative initially, but proven later in the thread) about the RaspiCam locking the SD card file system as it wrote video to the disk. This blocked the logging to disk, causing the 0.75s blockage in the code loop.

Three different solutions / workarounds were suggested all of which I’ve now implemented.

  1. The first is to reduce the video from 1080p at 30 frames per second – I’ve now set it to 720p at 15 frames per secind and with a corresponding lower bit rate which still meets Vimeo’s requirements for an HD video.  This will reduce the delay time due to less data being written, but it will still be there.
  2. The second was to redirect the logging output to the shared memory file system in RAM (/dev/shm) – that way the RaspiCam disk write shouldn’t block the logging RAM disk writes.  Once a flight ends, the log file is copied from the RAM disk to the SD card.
  3. Finally, I added code to prevent swapping code to disk – the code is now always held in memory, so this shouldn’t be affected by the RaspiCam video write.

This is a belt and braces approach, and I suspect that just the second solution was necessary, but the other two could well increase performance and stability of of the code marginally.  I’ve run the changes in a simulated flight by not powering the motors, and certainly the code had no problems, and there might have been a marginal increase in loop speed, so I’m looking forward to the next live test to see if it works.  Once more, I’m praying for good weather.

All I could do was watch in horror…

All I could do is watch in horror! from Andy Baker on Vimeo.

Net result, 3 broken blades (one in a previous flight and two in this).

Prior to that, I’d had several good flights, but somehow taking video and a good flying were out of sync.  Even in the one above, I’d turned off the “3rd person” video by accident as I saw the crash approaching, so only Phoebe’s point of view remains.

The drift into the railings is the breeze’ fault – in the hour I was testing, it went from still to a few miles an hour coming from behind Phoebe – hence the crash into the railings – completely the opposite directions to where she normally drifts.  She is clearly very wind-sensitive, and that definitely raises the game on sorting out the lateral drift code to compensate.  At the same time, it does suggest that if I am extremely careful to make sure the take-off happens with the motor tips as horizontal as possible, there is a good chance of no drift and a vertical takeoff – and that raises the option of moving the testing into the garage during winter.  Still scares me a bit though!

One thing that struck me (metaphorically, thank heavens), is that a vast amount of power is used for each flight.  First flight on a full battery levelled at 5 feet, next at 4 and the next at 3.  So I recharged the battery for the next flight, which is the one on video.

I have one set of blades left, but have just purchased some more – the current ones at 11 x 5 (11 inches total blade length x 0.5 inches pitch); the new ones are 10 x 4.5 – I’m hoping this will reduce the friction imposed on the blades and hence the power they take from the battery.  The flip side is they’ll need to spin faster to take off, so I’ll need to do some more minor tinkering on take-off.  They should work fine though as it’s this style blade that comes in the DJI F450 that was my starting point for this project.

Could it get any worse?

Yes – a broken arm, and suspected motor damage.  The arm is £5 and in stock which means I can do most of the rebuild quickly, but I could be waiting for weeks for the motors – not happy and luckily I seem to have found another guy in Poland who has stockpiled these motors – so glad Poland joined the EU!.

Having checked the stats, I know an extremely over-enthusiastic pitch PID d-gain (by a factor of 400!) caused the problem – so entirely my own fault.  D’oh.

Broken arm

Broken arm

On the plus side (gotta stay positive or I’ll go stark raving mad!), the blade just did some lawn mowing, so is fit for reuse when the new arm and motors arrive next week!

P.S. Any know any witches / wizards / magicians / anyone else who can make an Indian summer come next week?  I need to get this take-off sorted outside before I can move indoors to continue fine tuning over the winter.

No Raspberry Pis were hurt in the making of this film…

but once more the blades and the gravel had a major fight, which again, the blades lost!

I think there’s more work to be done here! from Andy Baker on Vimeo.

On the plus side, the blades didn’t slice the LiPo battery power cable, so no risk of explosion – you can see me in protective glasses and gloves this time just in case.

That then means I have diagnostics to interpret, but even without those, just the video itself suggests the pitch PID has too high a gain, meaning it overshoots it’s correction leading to ever increasing front / back rocking.  My code won’t cope with > 90 degree tilt so when the rocking goes over 90 degrees, the drone flips onto it’s head and accelerates hard towards the gravel.

Two obvious solutions spring to mind:

  • reduce the pitch, roll and yaw PIDs’ P gains and / or add a negative D gain to each
  • add support for > 90 swings – this is harder and unnecessary at this level of flight control if fixing the gains works.

Still not sure about the yaw though – far higher than expected still – I think I need to use the yaw angle PID’s I output to infuence the motors much as I do already with the vertical speed PID. But I’ll leave that for another day.


Always look on the bright side of life…

… so the drone flew today.  Sadly, it also had a bit of a crash, so I didn’t get any video, but I do have some post crash photos of the setup and the damage:

Test setup

Test setup


Post-crash drone

Post-crash drone

On the bright side, it took of in a controlled manner and obtained a hover.  On the minus side,  I think I got my yaw compensation the wrong way round, meaning the drone span on the z-axis quicker and quicker, until the restraining strings started to tie up in a spiral, ultimately causing the drone to flip over and hit the gravel very hard.  As a result, the blades are severely trashed (gravel and high speed carbon blades don’t mix) and before they were totally shredded, they cut through the power cables from the LiPo battery.

Again, on the bright side, the battery didn’t explode, and still holds 12.4 volts, so I just need a new Deans connectors – I’ve already taken the old ones off, and covered the bare cables with grey heat-shrink tubing to prevent them shorting in the mean time.  Oh, and new blades.

Finally on another bright side, the new dangling-between-the-legs antenna worked beautifully, with a great WiFi SSH connection to the main RPi via the TP-Link power-line adapter plugged into mains power inside the garage.

So live testing will resume shortly when I have replacement blades!


Reality bites…

and has nasty, big, pointy teeth!

Live testing stats

Live testing!

This live action test shows the drone take-off well under manual control, and then throw itself manically at every piece of furniture it could reach. Luckily due to the restraints fitted onto the test slab (rubber bands), that furniture was a metal table frame. The drone lost the fight. One motor and blade to be replaced, but I knew this would happen so already have spares standing by.

What did I learn? That my various PID settings were out by a factor of 10, and that there was a bug in my code for transitioning between manual take-off and auto-hover which meant the z-axis PID differenial was huge, which was what instigated the giant spikes! From that point on the rest was history! Ouch!

What’s up, Doc?

Don’t ask me, I haven’t the faintest idea, but when I run the drone in simulation mode (without power to the rotor blades, but otherwise identical), all works fine as per yesterday’s spreadsheets, yet once I have power to the blades, I get this mess, and yet there is no direct feedback from the blade motors to the Python code.  The only direct connection between the two is the power supply, but if there were problems there, I should see a lot worse symptoms than this spreadsheet shows (like a reboot!).

Catastrophic random shutdown of blades!

Catastrophic random shutdown of blades!

It’s clearly the z PID outputs that are collapsing catastrophically, yet there is no reason why. Currently, I am flummoxed!