Excel’s pants

I want to make a 3D graph from my flight stats i.e. given 3 columns, says X, Y and Z from the ground facing video (x 2) and down-facing LiDAR, I want a 3D picture.  It turns out you can’t just plug in these 3 columns into a 3D surface chart, you need a mesh which Excel doesn’t support directly, so you have to purchase another app; the first one I stumbled across was called XYZ MESH which cost $90 for a single user.  So instead I’ve done a simple version with 85 lines of python code.

Python Excel Mesh

Python Excel Mesh

These are the results of another square flight this morning.  There’s lots more that can be done such as working out how XL chose these colours, and getting the points to join up, but this is the hard bit done.  Here’s the code.

import csv

min_x = 0.0
max_x = 0.0
min_y = 0.0
max_y = 0.0

with open("3DSquare.csv", 'rb') as fp_csv:
    fp_reader = csv.reader(fp_csv)
    row_num = 0
    data = []
    for row in fp_reader:
        if row_num == 0:
            column_num = 0
            for column in row:
                if column.strip() == "qdx_fuse":
                    x_index = column_num
                elif column.strip() == "qdy_fuse":
                    y_index = column_num
                elif column.strip() == "qdz_fuse":
                    z_index = column_num
                column_num += 1    
            print
            print x_index
            print y_index
            print z_index    
        else:
            x = float(row[x_index].strip())
            y = float(row[y_index].strip())
            z = float(row[z_index].strip())

            if x < min_x: min_x = x if x > max_x:
                max_x = x
            if y < min_y: min_y = y if y > max_y:
                max_y = y

            data.append((x,y,z))

        row_num += 1


x_increment = (max_x - min_x) / 250        
y_increment = (max_y - min_y) / 250        

print row_num        

map_dict = {}
x_dict = {}
y_dict = {}
for (x, y, z) in data:

    x_index = int(round(x / x_increment))
    y_index = int(round(y / y_increment))


    x_dict[x_index] = True
    y_dict[y_index] = True
    map_dict[(x_index,y_index)] = z

lines = []
line = ""
for x in sorted(x_dict.keys()):
    line += ", %f" % x   

lines.append(line + "\n")   

for y in sorted(y_dict.keys()):
    line = "%f" % y

    for x in sorted(x_dict.keys()):
        if (x, y) in map_dict:
            line += ", %f" % map_dict[(x,y)]
        else:
            line += ", "

    lines.append(line + "\n")

with open("mashed.csv", "wb") as mash:
    for line in lines:
        mash.write(line)                

The reason I want this will become clear tomorrow as part of my last post for a week while I’m at Disney.


Ah, that looks better…

Way cooler!

Way cooler!

Autonomous Autopilot, Almost

So the autopilot is now full autonomous, able to direct the main motion processor between various intermediate GPS waypoints to the final destination.  It will soft-abort flights via a controlled immediate landing if objects are in the way, the number of ‘visible’ satellites drops below a safety limit, or at the end of the flight.

Or it would if my GPS receiver / the weather would behave.  It’s only ever got 11 satellites max, and today it was struggling to keep 8.  Roughly 10 seems to be the acceptable lower limit my Mavic will accept for GPS control and today, it had no problem finding 11+.  Hermione on the other hand could only see 8 at the highest and this often drops mid-flight.  Without a constant set of satellites the position drifts – I’ve seen 40+m errors today.  Luckily I have code to abort a flight if the number of satellites drops below the minimum, and that’s what I’ve seen; in addition, the GPS is used to produce the direction of travel, but not the speed which is fixed at a gentle 0.3m/s to allow the down-facing video to track lateral motion accurately.

Here’s the GPS data I got for my standard 2m square flight; as you can see the return to base which in real life was very accurate, was 4m out by GPS’ account:

GPS accuracy & resolution

GPS accuracy & resolution

So I need the skies to clear and ideally to find a better GPS receiver, and herein lies the problem: I’m off to Disney Land, Paris next week.  Sod’s law says GPS conditions will be perfect here in England next week!


P.S. Just found myself a USB receiver that covers both GPS and GLONASS. GPS is USA owned and has 31 satellites; GLONASS is the Russian equivalent with 24 satellites; hopefully together I’ll be able to get much higher number of satellites, and hence much more accuracy as a result. Sadly, the proof of this pudding will only happen after a week of Disney junk-food eating. 🙁

Yaw(n)

I think I’ve finished writing the code to support GPS tracking: a target GPS location is stored prior to the flight; the flight takes off pointing in any direction, and once it has learned it’s own GPS takeoff position, it heads towards the target, updating the direction to the target each time it gets a new GPS fix for where it is, and finally landing when it’s current position matches the target GPS position.

There’s a lot of testing to be done before I can try it live.  The new autopilot code is pretty much a rewrite from scratch.  It now has multiple partial flight plans:

  • take-off
  • landing
  • file defined direction + speed + time
  • GPS initial location finding
  • GPS tracking to destination
  • Scanse abort.

It switches between these depending on either external Scanse or GPS inputs, or the flight plan section completing e.g. the GPS flight plan reaches its destination or the file-based flight plan has completed the various time-based lateral movements and hovers (both of which will then swap to the landing flight plan).  Luckily, most of this testing can be carried out passively.

The first test preempts all of these; as per yesterday’s post, I should be able to use my magnetometer combined with its magnetic declination to find Hermione’s initial orientation compared with true (i.e. GPS) north.  At the same time, I could use the magnetometer to provide long term yaw values to fused with the integrated yaw rate from the gyrometer.  Here’s what I got from two sequential passive tests:

Compass & integrated Gyro yaw

Compass & integrated Gyro yaw

I’m not hugely impressed with the results of either.

  • The difference between compass and integrated gyro yaw doesn’t match as tightly as I was expecting – in its current state, I won’t be using the compass direction as a long term fuse with the integrated gyro yaw unless I can improve this.
  • The blobs in the lower pair are compass orientation values as she’s sitting on the ground – half way through I rotate her clockwise roughly 90°.  The rotation angle is pretty good as are the direction (NW -> NE -> SE, but I don’t like the distributed density of the blobs as she sat still on the ground at each location – I think I’ll have to use an average value for the starting orientation value passed to the autopilot for GPS tracking processing.

Magnetic Declination

The one thing holding me back with GPS tracking is the direction Hermione is pointing at launch; in particular, she can find out the flight direction to the target based on the difference between the GPS positions of takeoff and landing points, but Hermione’s compass only gives orientation angles based on magnetic north, and the difference between true and magnetic north varies over time and space.

After digging around, it turns out “Magnetic Declination” is the name of the angle between magnetic and true north, which for me is currently -1° 5′ i.e. at takeoff, when Hermione’s compass says she’s pointing magnetic north, she’s actually point +1° 5′ from true north.  For me, that means it’s irrelevant for the short distance flights I’m implementing.

In the perfect world, I would like to have a magnetic declination table, mapping GPS longitude / latitude to magnetic declination angle, or use a GPS receiver which has one of these built in (they do exist apparently), but for now simply knowing the magnetic declination is negligible for my few meter flights is enough.

Just FYI, Cambridge’ magnetic declination is -0º 30′ which is even more negligible!

2m square

While waiting for half-decent weather, I’ve been tinkering and tweaking to make sure everything is as good as possible before moving on to the GPS tracking code. This is the result: a 2m square.

As a result, I’ve updated the code on GitHub.

Global Warming

In early spring this year (March – mid-April) conditions were good for flying; light stable winds with sunny days and rainy days all forecast accurately days in advance.  The net of this was lots of test flights, lots of videos, and the only downside was the need to put Hermione’s HoG RPi B3 and PCB in a customize Pimoroni Tangerine PiBow to try to keep temperatures in the IMU above 20°C where its sensor offsets and gains are most stable.

In the second half of spring, the temperature soared to the high twenties / early thirties; there was no breeze and no rain; the weather was Mediterranean midsummer.  The lawn was starched and featureless for the down-facing lateral-motion tracking RPi camera and so flights moved to the gravel drive; more damage to the props ensued but progress was good.

It’s now nominally mid-summer here.  The weather’s moved to blustery showers, torrential rain storms, wind speed in the teens and sometimes gale force, with bursts of thunder and lightening and to make it worse, it changes every half an hour.  The temperature is mid-to-high teens. It’s standard autumnal weather in all but name, and it’s impossible to fly in; I get my kit out in bright sunshine and low twenties with no breeze, and by the time it’s set up, the sky has clouded over threatening rain, the wind has risen into the teens and the temperature has dropped into the teens.

Why am I telling you?  Because now, I’ve pretty much finished the GPS routing branch code, and in doing so, I’ve spotted a couple of fixes / enhancements I’ve back-applied to the main line; if I’m right, I may have fixed the feature where Hermione always faces the way she’s going rather than the direction she was pointing at takeoff: to change her direction of flight, she continues to fly ‘forwards’ from her point of view but she yaws her body round to point her nose in the new direction.  Completely pointless really, just another challenge to tick of the list.

Here’s to decent English summer weather again so I can post something that might actually be interest to you, dear readers!

Between a rock and a hard place, again!

There’s been no progress on GPS routing in the last few days.

For RPi B3 performance reasons, I’m limited to down-facing video resolution of 320 x 320 pixels @ 10fps.  That rules out flying on the back lawn; only the gravel drive in sunshine* will do.  On the plus side, there’s a lot less clutter and more space on the drive once my wife’s gone to work in the morning.  On the downside, any negative flight result results in broken props.

Props

Props

I’ve only recently swapped from the T-motor CF to T-Motors white beechwood props, based on price, looks and sturdiness.  And this week, I’ve found out that the lovely white beechwoods are out of production.  It turns out I’d bought the UK’s stock last month, and, as a result, I’ve bought Germany’s (perhaps the EU’s) leftover stock this week.  As a result, these white props are in safe storage for show-and-tell at the Cotswold Raspberry Jam only; for testing, I’ve found these XOAR 12×4 beechwood props that are nearly identical, except they come in beechwood brown unsurprisingly.

These new props, combined with improved code changes have been tested, and as a result, GitHub has been updated.

Time now to finally get on with GPS routing, weather permitting – the forecast says rain for a week 🙁


*I am strongly considering adding down-facing bright LED lighting to Hermione so that flying conditions are not so significantly limited by weather conditions.

GPS routing design

Starting requirements:

  • a prerecorded GPS target location (and possible intermediate waypoints) saved to file
  • the takeoff GPS location
  • an arbitrary takeoff flight direction akin to an aeroplane taking off down a runway i.e. autopilot sends motion process velocity vector of (0.3,0) = 0.3m/s forwards from Hermione’s point of view.
  • autopilot receives updates from GPS when there is a change from the previous one (i.e. GPS resolution is roughly a meter, it’s rate is typically 1Hz, and the flight speed is fixed at about 0.3m/s so each GPS update won’t necessarily be a change from the known location)
GPS routing

GPS routing

With the above, when autopilot receives a changed GPS location update, a GPS direction of flight (θ1) is calculated based on the difference between the previous (L1) and current (L2) GPS location.  Another direction (θ2) is determined from the current (L2) to the target GPS locations.  Knowing these two directions – the direction it is and should be going – autopilot sends motion processing a yawed version of the current (initially (0.3,0)) velocity vector.  This repeats until the current GPS location matches the target GPS location, at which point, a standard landing is sent by the autopilot to the motion processor – or in future, another waypoint is read from file and the above repeated again.

The above is relatively simple to implement, however there’s a lot of finicky bits to shuffle around so the code works:

  • Auto-pilot needs to be told it’s using GPS not file based routing
  • GPS waypoints are collected by the main process
  • flight GPS locations are processed by the autopilot
  • GPS is started fresh each flight / waypoint collection as the GPS process feeds into different processes per-flight or waypoint
  • Autopilot takeoff hovers while waiting to acquire enough satellites at the starting position, only then triggering the movement along the “runway”.

There’s probably others I haven’t thought of yet.

The result will be a bit of a zig-zaggy flight towards the target, primarily because the GPS target is likely to be less than 10m away, and GPS resolutions is down at about 1m.  If I get this to work, this is the last but one step of full-autonomous-flight completed!  However, the last is perhaps the hardest (perhaps impossible) to achieve: the route through a maze.

Hello pcworld.com readers!

First, I have to apologise on pcworld.com’s behalf – they’ve republished an article from over 2 years ago without bothering to update it in any way – they didn’t even bother to check the links to here still existed, so well done y’all for making it here!

Luckily the world has moved on here since then.  Have a browse, watch the videos, but I would strongly advise you not to follow in my footsteps – here’s why.

Cheers,

Hove