- Zoe works as best as she can, but she doesn’t have the performance to run the video above 480 x 480 pixels at 20 fps. This means lateral motion over the IKEA play mat works, but not on the gravel on the drive, never mind the grass. I have found a micro-USB GPS dongle which I may add but I’ve yet to find a micro-USB hub to allow power, WiFi and GPS to go into the single micro USB port (the other is blocked by the frame)
- Hermione can process video macro-blocks at 640 x 640 pixels @ 20 fps (and perhaps higher) which may be enough for the gravel drive, but I’m completely out of ideas for the source of the I²C errors from the Garmin LiDAR-Lite V3 when it is connected even if it’s not run.
I’ll continue working on the I²C to the end of the month, but after that I’ll be downing drone development and move on to other things, probably still Pi related nonetheless.
Here’s what I’ve captured from my scope.
It’s mostly at 2.5V with 3.3V peaks at 50ms or 20Hz corresponding to the LiDAR sampling frequency.
So the interrupt is kind of working, except the GPIO pull down is not ‘hard enough to get the 2.5V down to 0V. Because of that, the GPIO rising edge code isn’t capturing these tiny interrupt peaks.
This suggests the LiDAR output isn’t floating as the spec implies, but has a pull-up which is lower resistance than the GPIO pull down resistor. Seems I need to add an explicit pull-down resistor to aid the GPIO pull-down internal resistor. Time to find out what that is.
P.S. The internal GPIO pull up/down resistor is apparently 50k, and a quick calculation suggests that means the Garmin pull-up is about 17k – this results in the base-level 2.5V shown above. So I added a 10k between the interrupt and ground, as this would bring the ‘floating’ voltage to less than 50% and hopefully enough to allow the GPIO pull up/down to work. But it didn’t, it dragged everything down to near zero (including the spikes), so I removed it, but the spikes shown above had vanished, suggesting the 10k may have broken something. The Garmin is still producing height values luckily. Annoyingly, it had no effect on the I2C errors. I’m running out of ideas of what’s causing these. 🙁
I’ve flown my Mavic probably for 20 minutes over the course of 5 short flights simply to get familiar with the controls while dodging the rain showers of the last couple of days. I’m back inside again trying to track down why Hermione has started throwing her I²C wobbly again.
Motion processing is working well, keeping processing close to the minimum 100Hz regardless of other sensor inputs – here 156 samples were processed in 1.724s.
Garmin’s height is running stably at the intended 20Hz and it’s well withing the accuracy possible for distances less than 1m
Garmin LiDAR v3
Here’s the problem though: the IMU is fine for 862 samples averaged into the 155 motion processing blocks, showing just gravity as Hermione sits on the ground, but suddenly the IMU values spike for no reason for the 156 sample average. Note that this happens only when the Garmin is plugged in. There are in fact two spikes: the first is shown, the second causes an I/O exception and the diagnostics are dumped:
I’ve tried power supplies up to 3.4A, both battery and mains powered; I’ve resoldered various critical PCB joins; I’ve added the 680uF capacitor as the Garmin spec suggests despite Zoe being fine without it, and I’ve used a newly flashed SD card, all to no avail.
I have two things left to try:
- currently the Garmin is read every motion processing loop, despite being updated at 20Hz; the spec says there’s an interrupt, but as yet, I’ve not got it to work. Must try harder!
- Failing that, I’ll have to replace the MPU-9250 with another, and see if the current one is faulty.
Beyond these two, I’m out for ideas.
My intent with GPS and compass it that Hermione flies from an arbitrary take-off location to a predetermined target GPS location, oriented in the direction she’s flying.
Breaking that down into a little more detail.
- Turn Hermione on and calibrate the compass, and wait for enough GPS satellites to be acquired.
- Carry her to the destination landing point and capture the GPS coordinated, saving them to file.
- Move to a random place in the open flying area and kick off the flight.
- Before take-off, acquire the GPS coordinates of the starting point, and from that and the target coordinates, get the 3D flight direction vector
- On takeoff, climb to 1m, and while hovering, yaw to point in the direction of the destination target vector using the compass as the only tool to give a N(X), W(Y), Up(Z) orientation vector – some account needs to be taken for magnetic north (compass) vs. true north (GPS)
- Once done, fly towards the target, always pointing in the way she’s flying (i.e. yaw target is linked to velocity sensor input), current GPS position changing during the flight always realigning the direction target vector to the destination position.
- On arrival at the target GPS location, she hovers for a second (i.e. braking) and decends.
There’s a lot of detail hidden in the summary above, not least the fact that GPS provides yet another feed for 3D distance and velocity vectors to be fused with the accelerometer / PiCamera / LiDAR, so I’m going to have to go through it step by step
The first is to fly a square again, but with her oriented to the next direction at the hover, and once moving to the next corner, have yaw follow the direction of movement. Next comes compass calibration, and flight plan based upon magnetic north west and up.
However, someone’s invoked Murphy’s / Sod’s law on me again: Hermione is seeing the I2C errors again despite no hardware or software changes in this area. Zoe is behaving better, and I’m trying to double the motion tracking by doubling the video frame rate / sampling rate for the Garmin LiDAR-Lite; the rate change is working for both, but the LiDAR readings see to be duff, reading 60cm when the flight height is less than 10cm. Grr 🙁
Before moving on to compass and GPS usage, there’s one last step I want to ensure works: lateral movement.
The flight plan is defined thus:
- take-off in the center of a square flight plan to about 1m height
- move left by 50cm
- move forward by 50cm – this place her in to top left corner of the square
- move right by 1m
- move back by 1m
- move left by 1m
- move forwards by 50cm
- move right by 50cm
- land back at the take-off point.
The result’s not perfect despite running the ground facing camera at 640 x 640 pixels; to be honest, with lawn underneath her, I still think she did pretty well. She’s still a little lurchy, but I think some pitch / roll rotation PID tuning over the IKEA mat should resolve this quickly. Once again, you judge whether she achieved this 34 second flight well enough?
Zoe is now maxed out; any increased video frame size above the current 400 x 400 pixels at 10 fps leads to a IMU FIFO overflow i.e. the video processing simply takes too long. She’s also run out of physical space on her frame for more sensors.
On the other hand, Hermione has loads of physical space, but is plagued with I2C problems on her A+.
To add GPS for flight plans, and Scanse Sweep for object avoidance, I need more cores. For the moment that means a B2 or B3. Currently I’m leaning towards a B2 as I don’t need the 64 bit kernel of the B3, nor the built in WiFi or Bluetooth – I’d rather continue to use a faster WiFi USB dongle. With the extra cores, I can move the video processing out of the motion processing into a different process, and have it feed the latest values to the motion processing when available. Hopefully that will mean support for higher video frame size and rate and perhaps also increase the IMU sampling rate back to the 1kHz – I’ve had to reduce it to 500Hz currently. Having the extra cores and 4 USB ports means GPS and Scanse Sweep should be much easier to add – I doubt the A3 (if it ever appears) will support those extra USB ports.
So it’s a B2. For the sake of up to date build instructions, I’ll be installing her from scratch and blogging the instructions once complete.
Zoe is now running my split cluster gather + process code for the RaspiCam video macro-blocks. She has super-bright LEDs from Broadcom with ceramic heatsinks so the frame doesn’t melt and she’s running the video at 400 x 400 px at 10fps.
And this peops, is nearly a good as it can be without more CPU cores or (heaven forbid) moving away from interpreted CPython to pre-compiled C*. Don’t get me wrong, I can (will?) probably add minor tweaks to process compass data – the code is already collecting that; adding intentional lateral motion to the flight plan costs absolutely nothing – hover stably in a stable headwind is identical processing to intentional forwards movement in no wind. But beyond that, I need more CPU cores without significant additional power requirements to support GPS and Scanse Sweep. I hope that’s what the A3 eventually brings.
I’ve updated everything I can on GitHub to represent the current (and perhaps final) state of play.
* That’s not quite true; PyPy is python with a just in time (JIT) compiler. Apparently, it’s the dogs’ bollocks, the mutts’ nuts, the puppies’ plums. Yet when I last tried, it was slower, probably due to the RPi.GPIO and RPIO libraries needed. To integrate those with pypy requires a lot of work which up until now has simply not been necessary.
Both flights use identical code. There are two tweaks compared to the previous videos:
- I’ve reduced the gyro rate PID P gain from 25 to 20 which has hugely increased the stability
- Zoe is using my refined algorithm for picking out the peaks in the macro-block frames – I think this is working better but there’s one further refinement I can make which should make it better yet.
I’d have liked to show Hermione doing the same, but for some reason she’s getting FIFO overflows. My best guess is that her A+ overclocked to turbo (1GHz CPU) isn’t as fast as a Zero’s default setting of 1GHz – no idea why. My first attempt on this has been improved scheduling by splitting the macro-block vectors processing into two phases:
- build up the dictionary of the set of macro-blocks
- processing the dictionary to identify the peaks.
Zoe does this in one fell swoop; Hermione schedules each independently, checking in between that the FIFO hasn’t filled up to a significant level, and if it has, deal with that first. This isn’t quite working yet in passive test, even on Zoe, and I can’t find out why! More anon.
A few test runs. In summary, with the LiDAR and Camera fused with the IMU, Zoe stays over her play mat at a controlled height for the length of the 30s flight. Without the fusion, she lasted just a few seconds before she drifted off the mat, lost her height, or headed to me with menace (kill ensued). I think that’s pretty conclusive code fusion works!
Currently, getting lateral motion from a frame full of macro-blocks is very simplistic: find the average SAD value for a frame, and then only included those vectors whose SAD is lower.
I’m quite surprised this works as well as it does but I’m fairly sure it can be improved. There are four factors to the content of a frame of macro-blocks.
- yaw change: all macro-block vectors will circle around the centre of the frame
- height change: all macro-blocks vectors will point towards or away from the centre of the frame.
- lateral motion change: all macro-blocks vectors are pointing in the same direction in the frame.
- noise: the whole purpose of macro-blocks is simply to find the best matching blocks between two frame; doing this with a chess set (for example) could well have any block from the first frame matching any one of the 50% of the second frame.
Given a frame of macro-blocks, yaw increment between frames can found from the gyro, and thus be removed easily.
The same goes for height too derived from LiDAR.
That leaves either noise or a lateral vector. By then averaging these values out, we can pick the vectors that are similar to the distance / direction of the average vector. SAD doesn’t come into the matter.
This won’t be my first step however: that’s to work out why the height of the flight wasn’t anything like as stable as I’d been expecting.