Hermione’s and Zoe’s powerware

All of Hermione and Zoes’ powerware – battery, ESCs, motors and props come from a single supplier: Paul Maddock who runs electricwingman.com based in Chester, UK where I used to live until 2010.  Over the years, he’s provided a great service, including a couple of specials for me, and so is definitely worth a mention.

Here’s the list of Hermione’s parts I have from him:

This is probably the best set of powerware possible, to match Hermione’s best frame possible.

I also got Zoe’s powerware and frame from there too, though annoyingly, her motors are no longer available:

Once more, THBAPSA.

Exploring negative G ‘errors’ from the IMU

I’ve been asking myself various questions and checking the answers through test flights:

  • Do the negative G values show up in passive flights?  If the motors aren’t powered up, negative G values are not seen.
  • What about alpf?  With alpf set to 0 or 1 (460 or 184 Hz accelerometer low pass filter), I see the negative G ‘error’, but set to 2 (92Hz) or greater it doesn’t happen.
  • What are the values of the other accelerometer and gyro readings when this happens with alpf set to 0?  These are all believable: ax = -4214, ay = -696, az = -270, qx = 2922, gy = 870, gz = 1531 – no magic hex numbers in there suggesting problems.
  • What about sampling frequency?  With alpf set to 0, but with the sampling frequency dropped from 500Hz to 200Hz (SMPLRT_DIV 1 => 4), there are still negative G ‘errors’.
  • What happens if the range is set to ±4g instead of ±2g with alpf set to 0?  The problem remains – there are negative G ‘errors’.

To me, that suggests that somehow, the negative G readings are not an ‘error’ but are real.  They are filtered out by the lower low pass filter frequencies.  The MPU-9250 is working perfectly.  I’ll continue running with sampling frequency of 500Hz, and alpf of 0, and with the diagnostic / protective code disabled.

This then points the blame for head-butting the ceiling towards the ESCs – it’s as though they have latched at maximum power.  The PWM signals I’m sending range from 1ms to 2ms inclusive, and I’m now wondering whether sometimes, the PWM output does hit 2ms and latches the ESC; I’ve turned down the maximum to 1.999ms and see what happens – it’s a hunch based on a long lost memory of something I’d seen or read.

Thinking outside the box

I’ve now remembered the best solution for insulating the sensor from the cooling breeze of the props – a blob of blu-tack on top of the sensor does the job perfectly.  I’d done this with Phoebe ages ago, and then forgotten!  Sure enough, Zoe flew several low drift flights right up until the last flight where she was hovering nicely with low drift when suddenly she changed her mind, whizzing upwards at full speed and smashing into the lounge ceiling!  Zoe’s been running this code for weeks now, and this is the first time something like this has happened to her.

But it had happened to the rebuilt Phoebe less than a week ago which makes me suspect a hardware or firmware bug rather than software; my best contender currently is a firmware bug shared by the T-motor Air ESCs both Zoe and Phoebe are now using.

Having said that, it’s now bound to be my fault; either way though, I’ve had to cancel all her public performances until the cause has been found and fixed.  🙁


P.S.  There is one other possibilty: although I’m not getting I2C errors any more (I can see that with diagnostics),  I don’t know if I am still getting duff data errors as I’ve removed those diagnostics with the switch to the FIFO; if I get one of those, they’ll show up as Z axis acceleration as being negative (falling faster than gravity!), which would then explain the sudden unpredictable corrective increase in vertical speed.  I need to reinstate the code to protect against that and see what happens.

Squid

I’ve spent a lot of the last weak swearing while I try to bring Phoebe back to life. If I’d put 5 quid (“Squid” – geddit) in my swear box over the last week, I’d be a millionaire now.  But the physical costs are more that the swearing: so far, an A+, 8 ESCs, and an SD card have all died in the process of bringing Phoebe back from the dead.  And yet she’s refusing to work:

Squid

Squid

She flew once in “Octopus” format, rocketing up into the air at several meters per second before dropping like a rock once I killed the flight.  This was probably a symptom not the cause of the subsequent problems: if I plug the battery in while she’s like this, everything works fine – critically I can see the IMU, Barometer and URF over I2C at 400kbps.  As soon as I pop the other half of the frame on top, and plug in the motors, I2C dies and she can’t read her sensors.

For the moment, I have no choice but to keep swearing and trying to work out what’s wrong.

Why is Phoebe so manic?

Given that Phoebe and Chloe are so similar, why is their behaviour so different?  Chloe is like a Mum: calm, mellow and thoughtful; Phoebe is like her 3 year old daughter, bouncing around, screaming with delight right to the point she hurts herself and starts crying*.

I’ve been thinking about what could be through cause of this behaviour; the HoG’s are identical, which means hardware.

Motors:

  • Chloe’s has T-motor MN3501-16 12N14P (12 coils, 14 magnets)
  • Phoebe’s has T-motors MT2216-11 12N14P (12 coils, 14 magnets)

Because they have the same coil / magnet configuration, if they have the same ESC with the same PWM pulse width feeding it, they will rotate at the same speed, so this isn’t it.

Props

  • Chloe’s has T-motor 13 x 4.4 CF props – the larger of the two sizes recommended for use with 11.1 batteries and the motors (above) she has
  • Phoebe’s has T-motor 11 x 3.7 CF props – the middle of the 3 sets of props recommended for use with 11.1V batteries, and the motors (above) she has

So assuming they use identical PWM and ESCs and the motors above with identical coil / magnets, how much power to their props generate?  Dunno, but given Chloe weighs more than Phoebe, and has longer arms than Phoebe, there’s some logic that suggests she needs proportionally larger bigger props, which she does.  So this doesn’t seem a likely cause.

ESCs

  • Chloe’s has T-motor 30A opto ESCs
  • Phoebe has DJI 30A opto ESCs

The translation of the PWM pulse to the motors coil switching is very specific to each ESC – the details can only be found from the code running on the microcontroller inside the ESC so I have little to really prove it’s the ESCs fault except DJI have replaced the ESCs I use with new ones – perhaps related to Phoebe’s nutty behaviour – at least that’s plausible.

And last, but not least comes Noise

  • Chloe’s HoG is attached to the rest of her frame with very soft silicone grommets; there is no hard physically connect between her HoG and the motor / prop noise transmitted through the rest of the frame.
  • Phoebe on the other hand is directly connected to the rest of the frame and feels everything.

I already have silicone grommets on order (and they should have arrived today), so I shall be fitting those as soon as they arrive.

I may well also buy new ESCs for Phoebe that match Chloe, but I’ll need to check my bank account first as they’ll be £125 for 4!

Why bother?  Because Phoebe is running through props at probably 10 times the rate Chloe does, and so even a change in ESCs will pay for itself within a month!

P.S. While I was search for what 12N14P meant, I found this article about how to wind the coils of a 12N14P motor – worth a read if only for background knowledge.


*Through experience within earshot right know, the comparison I’ve made between Phoebe and Chloe, and my wife and daughter is absolutely accurate!

Is my code too fast?

Speculation, but I’m still on the search for noise…

The main loop in my code takes on average 6ms or 170 loops per second (Hz), during which the code

  • checks the commands for what Phoebe should do
  • reads the sensors
  • calculates critical angles
  • runs the PIDs
  • updates the PWMs

and I wonder whether this updating of PWMs at 6ms intervals is causing broad-spectrum noise from the blades depending on what the updates are?  Is it this noise forcing me to use the very low (5Hz) DLPF combined with the long complementary filter (0.5s)?

Have a look at the audio spectrum analyser results I got a while back.  Although there are clear peaks showing the blade RPM , these all should be filtered out by the DLPF.  But there is also a lot of other noise which wasn’t present before the blades started spinning.

If that noise is caused by the constant PWM changes, then I could apply another filter equivalent to the complementary filter, but applied to the change in the PWM signals for each blade, set to average with a tau of say 22ms (45Hz).  Certainly ESCs can accept PWM updates at this much lower frequency, and I suspect that although many advertise they can take updates up to 400Hz, perhaps they do filtering on their outputs to limit the noise rapid PWM changes would produce.  I can’t see inside an ESC, but it seems plausible.

Backing this up is my belief nothing in a quadcopter’s flight needs to react in 6ms – if nothing else, because the quad has weight and therefore linear and angular momentum, it simply can’t react in line with such rapid and significant changes.  Add to that the fact that unless the quad hits something, there’s nothing the quad needs to react to in that time frame, and it starts to sound plausible that actually, I need Phoebe to chill-out a bit by adding that filter.

For the moment, this remains a “Hmm, I wonder”, but if I can get decent behaviour from a DLPF of 5Hz, I may well consider upping the DLPF to 10, 20Hz or more, but soften the output via my own filter; I’d rather be in full-control of what’s going on than leave it to some inpenetrable magic box.

You may wonder “Why bother?”. First, as I said, I’d rather be in control not the DLPF; second, the DLPF is applied to gyro and acclerometer, and causes delay – this is unnecessary and inaccurate for the gyro; and thirdly, if I’m right, use of the DLPF is a sticky plaster to hide a flaw; instead the code should be tailored to fit the quad reactions better.

But first I need to prove that 5Hz DLPF solves the problem.

A glimpse into an ESC

although it’s actually more like an ultrasonic scan.

Slightly off-topic, but there was a comment conversation about whether ESC PWM input pulse width ratio was linear wrt output motors RPM.

I used the Audio Spectrum Analyzer app, combined with running tests at various PWM speeds.  Oh, and some Blue-Tak to keep Phoebe on the floor as she went over take-off speed.

Here’s what you get – make of it what you will, but it’s definitely not linear – although there are linear sections – the samples are at 5% PWM intervals and it is linear between 20 and 25%, 25 and 45% and finally between 45% and at least 60%.  I daren’t take the test further just in case the blue-tak let go and she whizzed up to the ceiling.  So the ESC microcontroller is doing some sort of logarithmic approximation – also not shown on the graph is that the motors RPM climbs steadily up to the PWM spin rate; the ESC is playing a role there to soften changes in desire spin rate.  I’m guessing the non-linearity of the ESC is to adjust / compensate for non-linearity of the power delivered by each blade for a given rotation speed.:

PWM vs RPM via ESC

PWM vs RPM via ESC

I guess if I have time to kill another day, I could attach my iPad iMSO digital ‘scope and see what the ESC output PWM looks like – but, as I say, that’s for another quiet day.  Now back to PID tuning, where depending on the weather, the next steps are to rerun the Z-N PID tuning with a better (nearer centre of gravity) hang of Phoebe, or a test flight with a lot lower Ki gain.

PID tuning II (lots of words, no pictures, but very interesting!)

I’ve been PID tuning for the past couple of days, and it’s been very tricky (for that read “effing PITA”).

When Phoebe had her centre of gravity dangling between her legs (Gentlemen, behave!), she was virtually self levelling – equal power to all blades and gravity would do the rest.

But with the rebuild, her centre of gravity, though still below the plane of the propellers, isn’t much below.  And that does make her very sensitive – a good thing as her reaction to disturbances will be super quick but only once I’ve got the PIDs tuned..

Thankfully, one of the reader’s, David, commented about mathematical tuning called the Ziegler-Nichols method.  It’s not perfect as this “we have a chip-on-our-shoulder about Z-N PID tuning and suggest you pay us for our tuner instead” article describes in detail.  However, the systems they name which should work match the dynamics of a quadcopter.  The Z-N algorithm takes away two thirds of the tuning, and makes very clear and simple what you’re looking for in the remaining “third”.  I strongly recommend you read it before carrying on reading here – it’s only a very small Wiki stub article.

The aim is to find the magical Ku, the ultimate gain, and the frequency of stable wobbles @ Ku. From that you can determined Kp, Ki and Kd for whatever kind of behaviour you might be looking for – mellow, jumpy or normal.  Sounds too good to be true, doesn’t it!

And indeed it is, but it’s no fault of Ziegler-Nichols that it doesn’t work reliably.  As my testing progressed, and the battery voltage reduces as a result, so do the resulting Ku values, and that can only be blamed on the ESC for not increasing the pulse widths of the PWM signal feeding the motors to compensate as it detects the reducing supply voltage of its battery.

The reducing supply voltage does change the speed of rotation since that driven directly by the PWM switching between the 3 phases of the motors coils.  But it does affect the power applied to those coils.

The ESC should be able to compensate for the reduced power based upon the fact motors have a manufacturer Kv value relating rpm to voltage supplied linearly. The ESC PWM signal to the motors is effectively dividing the input voltage from the batteries to a lower voltage. So if the ESC tracked the input voltage, it could increase the PWM pulse width to the motors to compensate. But I suspect my ESCs don’t do that leading to the every decreasing Ku values I’m seeing.

Now I’ve never liked the idea of a microcontroller ESC sitting between the Raspberry Pi and the motors – I don’t like the separation of control – it’s like Dalek’s serving tea to Winston Churchill in the second World War – you just can’t trust them. And I think now I understand enough about what’s going on that I could make my own ESCs that are better. Here’s the ingredients list:

  • Adafruit i2c 16-channel PWM servo driver which conveniently I have in stock
  • A rewrite of the PWM code to drive 3-phase PWM per motor (12 channel for four 3-phase motors)
  • 12 MOSFETs to switch the LiPo voltage across the motor phases according to the supplied PWM
  • 12 opto-isolators to keep the Raspberry Pi / Adafruit PWM safely isolated from the LiPo battery current across the MOSFETs / motors
  • An ADC converter tracking the voltage level of the LiPo where the value is used to increase the PWM pulse width in line with the decreasing voltage of the battery over time – this could also be used for an auto-landing / return home trigger for Phoebe as her batteries start running low.

The one missing piece of the puzzle is the connection between the ESC to motor PWM used to ‘reduce’ the power supplied.  But, again, thanks to another comment conversation with David, it should be possible through a combination of audio spectrum analysis to determine rpm, combined with the motor’s Kv to infer the ESC to motor PWM power on / off ratio.

I’m really quite excited about this, thanks David. Having written all this up, I definitely owe you a couple of beers.

Sadly, I do still need to sort out the PID tuning for the tetchy Phoebe, but this was an interesting diversion, and has got me thinking about how to make Phoebe the 100% Raspberry Pi Quadcopter controller I’d always hoped to achieve.  And that’s very exciting!

P.S. Just realized that the home grown ESC will need 2 PWM feeds per motors – the first provide the 3 phase output to a given motor at the required rotation speed, and the second uses Kv combined with the rotation speed to ensure the right voltage is supplied to the motors to run at that speed effectively / efficiently.

Audio Spectrum Analyser results

spectrum analyser output

Audio spectrum analyser results

I ran a quick test indoors playing with the audio spectrum analyser for my iPad.  Quite a cool piece of kit, though I did pay for a decent one.

The plot shows peaks at about ~65Hz, 126Hz and multiples of ~65Hz thereafter suggesting an RPM of about 3780.  Kv for my motors is 980 rpm / V which suggests 3.86 volts.

The PWM was set up with 1400us pulses – given the range is between 1000us and 2000us, this suggests 40% power from a freshly charged (12.2V) battery => 4.88V

To be honest, it doesn’t surprise me these two values don’t match.  In fact, I think Kv is as good as irrelevant for use with brushless motors, and in fact provides just another obfuscating factor to confuse the user:

Brushless DC motors work much like stepper motors. They are fed a 3 phase signal at a given frequency, and depending on the number of coils / magnets on the stator / rotor, the motor spins at a fraction of that 3 phase signal frequency. The amount of voltage driving the current they draw simply provides the power level required to drive the motors with a load (i.e. propellers) at the frequency defined by the 3-phase signal.

These days, the ESC uses a microcontroller to interpret the input PWM, and uses that simply as a factor to scale the motor rate from 0 to 100% motor spin rate.  The ESC drives the motors with 3-phase PWM-like digital signals. They use inductive feedback from the motors to track how much power (i.e. the width of the 3-phase PWM pulses) needs to be applied to the motors to make them spin at the desired rate, regardless of the load (i.e. propeller size / weight).

In the world of brushed motors where the magnets are stationary and the coils spin, I can see how Kv is relevant: the rate the motor spins is directly proportional to the voltage applied.  But brushed motors generate huge amounts of electrical noise, and wear out quickly, and so simply aren’t suitable for UAV’s (Unmanned Autonomous Vehicles – i.e. RC boats, planes, cars) of any kind.

IMHO, in the world of brushless motors, the mathematical value of Kv is frankly utterly meaningless, and only provides an arbitrary measure of what kind of battery can be used to provide suitable spin speeds for a given set of motors / propellers:

Kv of the order of 700 – 1000 are suitable for 3 cell / 11.1V LiPo’s with small propellers (8 – 10″)

As the Kv value drops into the say 300 – 700 range, then more cells are required to provide the same RPM – typically though, motors with these Kv values are intended for use with very big blades, where the required rpm to produce the same force is much less.

Obviously all the above is based upon my opinion, so may be utter bollox – I’m more than happy to be proven wrong.

P.S. On the plus side, the single sharp peaks at 65Hz intervals does suggest that all of my motors are rotating at the same speed, and none are obviously duff, which annoying means I still have no idea why yaw compensation is still plaguing Phoebe’s flights.

What the heck’s an ESC

ESC stands for Electronic Speed Control, but that’s not really what it does; what is does do is convert between the PWM signal from the quad controller software to the current driving the brushless motors.

How, you may ask?  Let’s look from both ends:

  • the PWM signal is passed to the ESC somewhere between 50 and 450 times a second.  The signal itself is a 1 – 2ms high pulse: 1ms means 0 power, 2ms means full power.  The RPIO.PWM allows for 1us steps in pulse size so there are 1000 steps between zero and full power.
  • The brushless motors have 3 coils of wire in the stator (the bit that doesn’t move), and there are many triplets magnets on the moving piece.  The motors are driven by a 3 phase signal over 3 wires – as the 3 coils are powered sequentially, so the magnets and therefore the motors move tracking the fully powered coil.

So the ESC converts this single phase digital pulse into 3 phases feeding the coils.  But it doesn’t end there.  If the signals powering the coils were sine wave in shape, then power would be turned into heat at any point the coils weren’t fully on or off, which would be a waste at best and dangerous at worst.  Those motors draw of the order of 10A, and the LiPo battery is 12V means the ESC may burn up to 120W of power if they working in an analogue manner.

So to avoid this, the ESC outputs just switch between on and off using MOSFETs  (Metal Oxide Silicon Field Effect Transistors) built in a simplified H-bridge configuration.  A full H-bridge is required for reverse movement, but since drone blades only spin the one way in fact the ‘H’ part of the bridge is not used.  By switching the power just on or off, then the MOSFETs are either dropping 0V or 0A and either way, waste little or no power.

That then means the speed of the motor is defined by the frequency of those pulses (and the number of triplets of magnets making up the motor), and the average power fed to the motor by the width of each pulse.  The ESCs use inductive feedback (or a 4th wire sometimes) to ensure the power applied by the pulse width is sufficient for the motors to spin at the pulse frequency.

But don’t forget it’s doing that for all 3 phases of the motors, and deriving it from a single simple PWM signal.  Which is why you should just buy an ESC (the bought ones have embedded microcontrollers to get this right) rather than try to do it via software or build one yourself.

Thought I’d share since it took me a quite a while a few months ago to work out what ESC stood for, what it did, and how before I realized I should just buy them!

P.S. The drone is going for a flight tomorrow – I’ll either post a video or a photo of me in tears depending on how it goes!