Analysing the analysis – a closer look at the Saki air base attack satellite imagery

Yesterday – or rather, in the early hours of today – I posted my last blog, Novofedorivka – Saki Air base attack satellite imagery – The aftermath.

In that blog I made a typo. For every word where I meant to say Su-24, I said Su-23. This included in the satellite imagery labelling. So how could this possibly happen as I knew fully that they were Su-24s? I’d called them this correctly in the blog before that and regardless – I know what a Su-24 is.

To add salt to the wound of the error, on my desk next to me at the time of doing the analysis, I had the excellent books by Yefim Gordon & D Komissarov Sukhoi Su-24 and Sukhoi Su-27 & 30/33/34/35. They were still on my desk in the morning when I got up. I’d had the idea on going into a little detail about the aircraft themselves, but changed my mind.

The books still on my desk in the morning.

Looking back at the creation of the blog, I’m pretty sure I know what happened. When I started working on the imagery, when I typed in the first Su-24 label, I inadvertently typed Su-23. This could have been in error by hitting the 3 key instead of the 4, or by just stupidly typing it incorrectly.

From there, the rest is history. I copy/pasted the label for the others in the imagery, and this is where the brain takes over. I subconsciously took in Su-23 as being correct – regardless of knowing what they were, and having pointers near me to correct the mistake (including checking back on the other imagery and blog looking for changes).

Moreover, when it came to proof reading the whole thing, it still slipped through the net again. I even found other mistakes that I rectified.

In other words I totally believed what I was typing and had typed was correct, even though subconsciously I knew it was wrong. And I let it pass – I was seeing what I wanted to see

In my daytime Air Traffic Control world we use the well known term confirmation bias for this.

What is interesting about the whole thing is that just two hours before, in a busy radar session, I was calling a couple of aircraft by the wrong callsign. This is extremely common for us, and for pilots too.

To explain. We have radar screens with data-blocks that show the aircraft callsign, altitude/level, selected level in the flight management system on the aircraft (via ADS-B) and the exit code from UK airspace or last two letters of the destination airfield. We have plenty of other things available to us via Mode-S, but these are selectable.

We also have electronic flight progress strips (eFPS) which has plenty more info on, but the callsign is the obvious one and what I want to look at here.

I can’t remember the exact callsigns, but take an example of EZY12QC – “Easy one two quebec charlie“. I called this one “Easy one two quebec golf” on its first contact, and despite having a eFPS and radar that i was fully interacting with, I continued to do so. It didn’t matter what was in front of me, it was “quebec golf”, not “quebec charlie”. There was at least another flight like that. All was safe as it was checked by the aircrew that the instructions were for them, but it adds extra workload and time to radio transmissions and getting the traffic moving.

An example of aircrew error is taking the wrong calls for other flights with similar callsigns – normally with the same airline, though inter-airline errors do occur. On one occasion, a flight I was working kept taking the call of another that was with the same airline. Eventually, after the fourth or fifth time, he apologised and said he’d been doing that flight the day before and couldn’t get it out of his head – despite coming from Spain and using the correct callsign up until then.

In ATC we use a combination of long term memory, and short term memory. The long term stuff is for things like procedures, sector frequencies etc. Airline callsigns come into this too – their actual airline callsign such as “Easy” for EasyJet, “Speedbird” for British Airways.

The short term stuff is things like co-ordinated agreements with other sectors, the actual traffic picture, flights on frequency etc.

Short term stuff we remove from our brains, once we have no use for it, but we keep the other stuff forever. I still remember things from RAF Lyneham when I was there in 1989!

And, of course, this isn’t an aviation thing. It is present in everything humans do in their lives.

So, how does this affect analysing imagery etc.?

With the last blog, it was probably a combination of being up since 7am, doing an afternoon shift finishing at 2200 UK time that included confirmation bias in the last hour – and then an hours drive home. In other words, a long, tiring day with a fuddled up brain already in place.

Going back to saying that we see what we want to see – analysing imagery has plenty of this.

Not everything of course, but occasionally it creeps in. And it happens to everyone.

I’ll take the Saki attack “aftermath” as a prime example of this as I think there’s several places this has happened. And I’m just going to say this now – this is not a direct dig at anyone in particular.

In fact, I’ll start with one of mine – or a possible one. I’ve been watching Saki since 2014 so know it pretty well I think. I also have access to some fantastic data on the base.

The two buildings destroyed at the revetments are known “workshops” used by the Russians for quick repairs to aircraft. Often this has entailed taking parts from one aircraft to put onto another to keep the fleet “airworthy”. This is likely why there was a Su-24 at the eastern building. Parts are stored in one of the revetments west of the building.

The two concrete parking areas also targeted were for vehicles, equipment and spare parts – often kept in boxes or crates. One has been referenced as a building in some analysis and on social media. This is completely wrong. You only have to look back through Google Earth history to see that often there are Su-24s parked there. But people are seeing what they want to see – and to be honest, being a little lazy and not checking themselves. It doesn’t take much to go back through GE history.

I have all this information stored in my head as long term memory and that is what I believe these areas are used for. At some stage over the last few months, and in particular over the last few days, these buildings and parking areas have become weapons storage areas according to reports and social media. Where this came from I have no idea, but certainly, since the attack they have been known as “ammo storage buildings”.

Likely, the main reason for this is because the number of boxes and crates has increased since the beginning of the war – and they’re green. My confirmation bias says these are all sorts of equipment, whereas others say they are ammo boxes because this is what they’ve read/been told; and their confirmation bias won’t say otherwise. Ammo boxes are being seen because they are green – and well, so are ammo boxes.

One of the concrete areas has white torpedo like objects. These are Su-24 3,000 litre external fuel tanks that they carry on the inner pylons, under the wings. In the aftermath imagery you can see they have been shifted by the power of the nearby explosion. These have been referenced to missiles in storage. They’re not.

In reality, we don’t actually know what was in these green boxes and crates. Logic tells me it isn’t all munitions as they have hardened areas specifically for this. But, the Russian forces do have open munitions storage areas located at bases all over the country so who is to say? More than likely, it will be a mix of things.

The real confirmation bias from this incident comes it at the main apron. The Planet imagery I provided for the morning before the attack showed three Su-24’s and three Su-30’s parked on the main flight line.

There are a number of things to note – referencing the first image below. Firstly, the aircraft follow white taxiway lines to a white square to stop and shut down. These squares are clearly visible where aircraft aren’t parked.

Secondly, next to each parking spot there is equipment used with the aircraft. Starter generators, wheel chocks, ladders and other things needed for the aircraft. These can be seen in between the parked aircraft in the imagery.

The last thing to note is that there isn’t an aircraft parked on the far west spot – this is the spot that in the post attack imagery there is supposedly a destroyed Su-24. As there’s no wreckage present, this can’t be confirmed 100%, but photos and video have been produced that do show a destroyed Su-24. Actually, in the post attack imagery the burnt area centres on the equipment between the parking spots.

Looking at the second image below you can clearly see all the equipment still in place. But many saw these as destroyed aircraft – and Hey, Presto! six more aircraft that are actually over to the east of the base have been destroyed!

Total confirmation bias – you are seeing what you want to see. Because we all want to see Russia fail (well not everyone, obviously).

And yet all the clues are there. At the probable destroyed Su-24 area, there’s a completely burnt out patch covering the parking squares – yet for the “other six” there isn’t. The obvious equipment – seen in imagery just 24 hours before – is ignored and declared as wrecks.

Whilst the aircraft that were on the flightline probably didn’t escape some damage, from confirmation bias we have claims that the whole fleet of aircraft were totally destroyed – and whilst it was a very successful attack – it wasn’t as successful as is claimed.

This leads to misinformation – and what I call ” Bad OSINT”.

The Snake Island attacks – Part Two

In my last blog, I tried to highlight the issues with analysing imagery and videos with only half a story.

I also tried to draw the attention to how fake videos can make one look at others with a lot of doubt as to whether they are real or not.

I concluded that more evidence was needed – in particular high resolution imagery from Maxar or Planet.

The good news is, that not long after the blog was posted, I was anonymously sent an image dated 7th May 2022 taken from either Maxar or Planet – the source didn’t say.

This clearly showed the wreck of the Project 11770 Serna class landing craft in the Snake Island harbour. It also showed the concrete blocks I wanted to see. This was useful as had the image been collected from before the attack, and there been no wreck, then at least the location was pretty much confirmed.

Even the blocks would have been enough then to conclude that the video was legitimate.

It wasn’t long after I received the image that it was published by AP, and shown on Twitter.

For those that don’t have Twitter access – Jon’s account is locked – here’s the image.

I also received a notification from a friend, Scott Tilley – well worth following on Twitter if you don’t. His satellite tracking capabilities and knowledge is fantastic.

His notification pointed me to a website that contained photographs of Snake Island – some of which depicted the concrete blocks used as the sea defences. A great find – and one that had slipped through my rushed searches.

So, hopefully this shows how information can take it’s time to get through to carry out a full analysis.

There’s reasons why the Intelligence services take their time over gathering data on incidents such as this.

Now, as further videos are coming through thick and fast of attacks on Snake Island, more confidence can be had over their legitimacy.

The wingman in this attack is probably very lucky not to have been taken out by the explosion created by the flight leader.

One has to question why the Russian forces are intent in staying at Snake Island. Their losses, I’d say, are greater than those taken by Ukraine.

My friend Capt(N) provided some information on the island in a recent Twitter thread. I’ve taken screenshots here as, again, not everyone uses Twitter.

The thread can be read here.

The Snake Island attacks – Part One

Whilst there is no doubt there have been several attacks on Russian equipment on Snake Island, in the last few days, some dubious video footage has been “leaked” on Twitter showing Russian ships under attack.

These videos do put into question those that do appear to be genuine.

For example, yesterdays – 6/5/22 – “news” that Project 11356M Admiral Grigorovich class FFGH Admiral Makarov was struck by multiple Neptune missiles immediately reminded me of the same claim against Project 22160 Corvette Vasily Bykov at the beginning of March, be it with MLRS weapons rather than the Neptune missiles.

I personally wasn’t convinced about the Makarov attack, and once further ridiculous Tweets materialised using ADSB data from FlightRadar24 (FR24) showing NATO aircraft “monitoring the situation” as proof that “something was going on” – well, I definitely didn’t believe it.

This is just poor “analysis” by people who haven’t got a clue what they’re talking about and should just not bother saying anything. Two examples below.

Unfortunately, the Ukraine war has brought out a substantial number of idiots that are suddenly “experts” in warfare, aircraft tracking, ship tracking and satellite imagery analysis. In reality, they are just plain fools.

And as Ben Kenobi says in Star Wars – “Who’s the more foolish? The fool or the fool that follows him?”

This is the problem with social media. These people have a “show” that they’re experts, and then they get thousands of followers that believe everything they come up with.

Personally, I don’t trust anyone with OSINT in the username.

This idiocy was highlighted when a video appeared, apparently from a TB2, showing Admiral Makarov on fire. This was clearly fake and taken from a video game – later identified as ARMA 3. One account on Twitter was able to recreate pretty much the same “video” in a matter of minutes.

So, whilst evidence of attacks are a good thing to have to assess whether losses have been taken or not – fake videos tend to sway people in the other direction.

Going back a few days, I believed the Project 03160 Raptor fast patrol boat attacks video from the 2nd of May – but the above now has me thinking otherwise. I did find it a little strange that the second Raptor hung around the area for so long, and didn’t really make much attempt to evade a potential strike. This highlights the problems with creating fake videos for propaganda – once one fake video appears, it makes others seem fake too – whether they are not.

Todays video of the Project 11770 Serna class landing craft being attacked at the Snake Island harbour area certainly got my “fake video” senses twitching when I saw it. Mainly because, by sheer coincidence, I’d obtained imagery of Snake Island from Capella Space, collected on the 4th May, and I’d taken a good look at the harbour area to search for any evidence of ship activity there.

Coupled with the potential fake videos from previous “attacks” one can start to see inconsistencies in this video.

One thing – I always say this regarding my analysis work – I can’t always be right. I like to be, and I take my time on it, but errors will creep in every now and again.

So let’s look at what I see in the imagery versus the video and I can lay my cards on the table with my thoughts – and as always, I’m open to any comments.

First of all, one link to the video on Twitter. It is also available on YouTube I believe.

A number things immediately grabbed my attention. It is visible even in the Twitter image above. All those blocks of squares and rectangles. They look like CGI – too perfect. That area gets pummelled by the sea most of the time. Granted, they could be containers just dumped into the sea, but I’m not convinced at this.

Also, the ramp to the sea looks too perfect – very straight lines, no sea lapping over it. The wall that runs along it, into the sea, is new.

Let’s look at some close-ups from the video.

This one above shows yet more blocks east of the ramp, and strange grooves, much like seating areas. No sea lapping over them.

The next two shows the same area from nearly directly above. Note the near perfect lines of the walls, and more importantly, these blocks again. What are they? Not containers. Maybe concrete block sea defences??

The next image gives an overall view of the harbour area. Note the blocks again, and the coastline itself.

Now let’s look at the Capella imagery.

Whilst not perfect – typically the worst part of the imagery is the harbour – the blocks in theory would stand out. There doesn’t appear to be any. The quality is enough to show the jaggedness of the rocks along the coast, but not much else. There does not appear to be a wall out to sea along the ramp – but this is inconclusive in this imagery.

We can move onto some hi-res imagery from Maxar, though I’m afraid to say I have no contract with them and so I have had to use images from elsewhere. Ideally, we could do with someone that does have a Maxar account – or Maxar themselves – to provide us with the high-res imagery.

The first is taken from a CNN article dated 14th March 2022 and states the image was collected on the 13th. I’ve had to zoom in a little for the screen grab.

Not ideal quality, but does it look like there’s been much of an upgrade to the harbour area? It doesn’t look like there has been. It’s hard to determine whether there are any blocks there.

The following image is reportedly from Planet, collected in the last few days, and published by Associated Press – AP. Whilst I couldn’t find a direct link, there’s plenty out there – for instance.

Moreover, searching in the Maxar archive, there has been a collection on 7th May 2022 which shows smoke coming from the building as shown above, just on the left edge. Note also the ship activity to the west of the island.

With these two images nearly aligning, we can conclude that the top image is very recent.

In my view, whilst there are small buildings near the harbour, one of which in the area east of the ramp – there appears to be no large blocks present. The wall into the sea by the ramp does appear to be present, but hard to determine whether it matches that in the video. It is still too hard to conclude from the imagery currently available whether the blocks are there or not.

Ideally now, then, we need that hi-res imagery that Maxar clearly has (note they’ve redacted the archive imagery of the island). Then we can put this one to bed once and for all.

Analysis isn’t just about seeing what is immediately in front of you. It is much, much deeper than that. Below sums it up nicely.

Just because it looks like Snake Island harbour in the video, doesn’t necessarily mean it is. You have to look at more than just the shape and the jetty.

Ironically, one proven event – the sinking of Moskva – is still to produce any video evidence that a missile attack led to its demise.

Say of that, what you want.