Real-Time Intelligence Interpretation Using UAVs (Unmanned Aerial Vehicles)
BY MAJOR (RESERVE) CHEN GRANAS
Visint, or visual intelligence, represents a broad range of intelligence gathering methods, from ground observations used since the dawn of history to advanced satellite photography at various frequencies. Visint is crucial to the creation and completion of the intelligence picture at all levels – tactic, operative and strategic.
Despite the importance of updated visual intelligence in battlefield management and decision-making, it is confronted with major obstacles due to technological problems related to film development and interpretation, or real-time communication and integration of information received from real-time observations.
The advantages of visual intelligence lie in the reliability and availability of information gleaned in such methods. Its drawbacks are low penetrability, the difficulty of sifting huge volumes of information and the weather sensitivity of most apparatuses.
In order to take full advantage of the visual dimension, one must ensure maximal recentness of the visint material produced. This capability is reached using real-time visint collection methods.
There are several techniques for producing real-time intelligence:
– Ground observations
– Aerial reconnaissance using a manned aircraft
– Aerial reconnaissance using an unmanned aircraft
The differences between those techniques lie in apparatus mobility, angle of observation, the sector under observation and the range of possible interpretation enabled by existing optical capabilities. While ground observation is static and encumbered by a low angle of observation, many obstructions to the view, and limited to a narrow sector, in terms of both range and field of vision, manned airborne observation uses higher angles, albeit at the cost of exposing the aircraft to threats. Unmanned aerial vehicles (UAVs), however, are less limited in terms of flight area, and can usually operate right above the target, observing it from various angles.
As for the optical quality of observation devices, although stationary ground observation can operate larger, high-quality devices, low-level atmospheric penetrability limits observation on most days of the year. While manned aircraft are big enough to carry high-quality payloads in relatively good atmospheric penetrability conditions, these airplanes require complex and costly stabilization systems. On the other hand, UAVs can carry only relatively smaller payloads, reducing their observation range compared to the other approaches.
With the modernization of airborne observation devices and the accelerated development of UAVs, the interpretation skills in this medium are also advancing. As opposed to classic airborne photograph interpretation[*], there now emerges a much “younger” profession: interpreting a live copy of Full Motion Video (FMV), namely, simultaneous interpretation of a dynamic image as it occurs; the interpreter is required to detect information and grasp its intelligence significance at one and the same time.
Until recently, the profession of interpreting a FMV was largely defined as an off-shoot of hardcopy interpretation, and the respective training courses looked very similar. The need for a redefinition arose from staff units and those operating real-time visint apparatuses, in view of the unique importance of their missions. The new definition has been reflected in the establishment of a UAV Professions School, which also offers a mission-related interpretation course for interpreters of UAV.
In order to analyze the interpreting profession, the following must be examined thoroughly:
– Understanding the question – What is interpretation, and what are the professional skills required?
– Analyzing the differences between interpreting a hardcopy, a softcopy and a FMV.
– What are the responsibilities of a FMV interpreter during a mission?
– The two previous items require a definition of the scope of knowledge required for such a professional.
___________________________________________________________________________
Visual Interpretation: Giving the image its intelligence significance, in terms of identifying components, defining its status and evaluating its relevance.
Intelligence Research: Appropriate contextualization of intelligence data in order to provide an accurate situation estimate.
The Interpreter’s Operational Dimensions
Time
When interpreting both hard and soft copies, a frozen image from the immediate or distant past is being interpreted. When interpreting a FMV, however, a dynamic image/situation is being interpreted, one which develops and transforms as it is being photographed.
Synchronous Decision-Making
When interpreting both hard and soft copies, the interpreter has “unlimited” time at his disposal, in the sense that s/he can view it for as long as necessary. While on a mission, there’s the added element of time pressure and the need to reach high interpretation capacities, which dictates time limits for the interpretation of each image. In practice, however, the interpretation work can be divided among several interpreters at any given moment.
A FMV interpreter is limited to the timeframe of the intelligence-significant event, which is can be instantaneous or of a very long duration. Based on their interpretation of the event, interpreters are required to report their findings and continue carrying out their mission, taking into proper consideration the data just interpreted (See Example in the box below). This requires high concentration over a long period of time, self-confidence and the ability to make real-time decision.
Synchronous Decision Making: Continuity
While tracking a car with a driver and no passengers, the car’s view was obstructed by a big building. Immediately afterwards, an identical car was spotted emerging from behind the building and surveillance continued as before. A few seconds later, the interpreter spotted the hand of a passenger outside the vehicle’s rear right window. Since the target vehicle was known to contain no passengers, the interpreter defined this car as mistaken, and immediately back-tracked to find the target vehicle. This was indeed identified in due course, and the mission proceeded as planned.
Required Knowledge
According to current practice, hardcopy interpreters specialize in a certain sector. The advantage of this approach is the interpreter’s familiarity with the sector, its weapon systems, recent deployment or infrastructural changes in the sector, and any other significant changes.
Softcopy interpreters enjoy the straightforward technical capability of retrieving a reference image (from a previous sortie, for example) in order to facilitate the detection of significant changes in the area of interest. As in the case of hardcopy developers and in view of the multiple interpreter work-stations available for interpreting softcopy material, it is possible for interpreters to specialize in certain sectors or mission types.
Conversely, a FMV interpreter analyzes with the image alone. Ideally, an additional interpreter works alongside. The complexity of maintaining operational aptitude for UAV interpreters dictates very limited numbers of interpreters, such that each is trained and qualified for all routine and emergency missions, in all sectors. This approach suffers from many disadvantages, but is inevitable in the present conditions.
The solution available for missions requiring a high degree of accuracy and expert attention to small details is working together with a sector expert who provides ongoing intelligence requirements during the sortie and holds a continuous dialogue with the FMV interpreter. This mission requires specific experience and a high degree of collaboration between the two. In order for this collaboration to succeed, high-quality intelligence aids and a preliminary, detailed and professional briefing must be provided.
Interpretation Techniques
A hardcopy interpreter usually works on a 3D (stereoscopic) image, which allows greater ease in spotting landmarks, fortifications and camouflages, and highlights suspicious elements. To interpret a hardcopy, one works top-down, from large to small details – orientation, looking for suspects and interpretation. Moreover, the hardcopy image has been photographed at a single location, vertical or diagonal as the case may be, but single all the same, so that any element obstructed by protrusion or cover will not be exposed in the photographs.
A softcopy interpreter usually works on a two-dimensional copy, although it is possible to create three-dimensional softcopies. In a 2D copy, it is pretty difficult to notice the small details of protrusions, fortifications and camouflage. On the other hand, it is pretty easy to manipulate the zooming ratios of photographs to enhance interpretive capability in searching for targets with a large optical signature in a distant zoom. In this case, too, interpretation will usually be top-down, with a single angle of photography.
In a live UAV copy, the photographed image can only be two-dimensional. The airplane’s constant motion actually changes the angle of photography at any given moment, which helps the interpreter’s brain to “construct” a 3D image of the photographed site. This necessitates continuous observation of the image and in-depth scanning of small details – fences, bumps, trees, buildings, suspicious points, trenches, etc.
The technology for creating a video image creates a situation where different details suddenly appear in the image following zoom changes. This stems from the camera’s resolution capability and the fact that it photographs in discrete pixels while the resolution limits are final. All this means that the FMV interpreter must work bottom-up – constructing the target image according to the details collected about it.
The last comparison category is the UAVs’ capability of observing a certain site from various directions, that is, their capability to determine an optimal angle of interpretation and to detect hidden elements in a steeper or shallower angle of observation, and in viewing directions which are more convenient for the interpreter.
Work Environment
Both hardcopy and softcopy interpreters work alone in their workstation, on readymade material that cannot be manipulated.
FMV interpreters, on the other hand, work in an environment wherein they are part of a flight crew carrying out a mission in an operational, real-time atmosphere, producing its output during that same mission, affected by all its limitations. This dimension has many additional ramifications, to be discussed below.
Three/Two-Dimensional Interpretation:
During an attack on a large, four-storey building, it was hit by a single bomb which caused it to collapse entirely in a symmetrical fashion. The controller, observing the attack by through an image transmitted from a real-time visual source, did not notice the change in the image and ordered that the attack be continued. The interpreter’s insistence that the target had been destroyed; an unnecessary second attack was prevented on the last minute.
|
Dimension |
Hardcopy |
Softcopy |
FMV |
|
Time |
Image frozen in a definite point in time |
Image frozen in a definite point in time |
Dynamic situation/image |
|
Synchronous Decision-Making |
“Unlimited” timeframe |
“Unlimited” timeframe |
Very narrow timeframe (limited to actual event) |
|
Required Knowledge |
Sector and weaponry specialization |
Mechanized reference |
Multi-sector, multi-system, no reference |
|
Interpretation Techniques |
3D |
2/3D |
2D basis, 3D construction |
|
Top-down |
Top-down |
Bottom-up |
|
|
A single angle of photography |
A single angle of photography |
Multi-directional photography |
|
|
Work Environment |
Independent |
Independent |
Teamwork |
Characteristics of a UAV Interpreter’s Work
As mentioned above, the UAV interpreter works as an integral part of the airplane’s flight crew, and comprises multiple functions:
– Studying and planning the mission in detail
– Operating the camera to obtain optimal images both by day and by night
– Navigating the camera and identifying the landscape
– Scanning the area for the required information
– Interpreting the image – giving it intelligence significance
– Distributing the information to various recipients
In fact, the real-time interpreter’s work can be seen as reflective of the classic intelligence work cycle – receiving updated queries, collecting pertinent information, researching and evaluating that information and distributing the output while redefining future queries in real time.
What distinguishes this kind of work is the fact that a live image interpreter conducts a sort of continuous intelligence research during most of his/her work. The need to conduct continuous research requires comprehensive knowledge and extensive professional experience, in order to make the most out of the information contained in the situation photographed as is occurs.
Queries -> Collection -> Research -> Distribution
 |
Follow-up/Responsive Queries
Real-Time Research:
While observing an abandoned installation, the interpreter noticed a typical behavior of a dog tied in the yard towards a person passing by. The interpreter, who owned dogs, identified it as a dog’s behavior when with its owner. He therefore decided to follow the person associated with the place by the dog’s behavior, and this surveillance did indeed reveal the location of the new installation.
Mission Planning
When planning a real-time intelligence mission, one must attend to details which may affect decision-making during the mission. When collecting routine intelligence, we make sure we have at our disposal as much information about the target as possible, detailed data about possible connections to targets in the area, information about the routine behavior of the target and his/its environment, and so on. In mission-related intelligence, in addition to the above, we must also stay updated about the operation’s details as it unfolds, about its procedures and contingencies. Understanding these data will help the UAV team to take the correct decisions in interpreting the target during the mission.
Camera Operation
As the previous examples show, interpretive nuances are crucial, as are quick responses to real-time events. These mean that the UAV interpreter must also be the one operating the camera. UAVs carry many types of camera for both day- and nighttime surveillance, each with its unique capabilities and calibrations, so that, in fact, the interpreter is also required to act as a technician specializing in several types of cameras, with different technologies. This expertise includes the ability to create optimal images in different working hours and varying atmospheric conditions.
“Optimal image” carries a very different meaning in different information contexts, both in terms of width of field of vision and in terms of the image’s contrast and brightness (in thermal imagery [FLIR- forward-looking infra-red] these calibrations are called “gain” and “level”). Professional sensor operation allows the interpreter to receive intelligence data even in problematic lighting conditions and in “saturated” areas hampering FLIR photography.
Camera Navigation and Orientation
The camera operator must be skilled in navigation using various devices and when in diverse landscape conditions in order to meet the requirements of all missions for which s/he is trained. This is in fact a two-pronged skill:
– Detecting a given reference point in optimal speed and precision (a house in a refugee camp, a tree in a forest).
– Locating a target found by the interpreter in the process of scanning the image (a reference point to a precision of a few meters, a house in a refugee camp).
If we complicate the task with dated navigation aids and weather disturbances such as haze and clouds, we can understand the requirement for a very high level of navigation skill. A tiny error in location can mean the difference between carrying out and canceling the mission, or worse, attacking the wrong target.
Scanning the Area in Search of a Target
In hard or soft copies, the interpreter scans the frozen, photographed image in search of the required information. In a live copy, the interpreter scans the image as events occur. Lifecopy interpreters use several scanning techniques:
– Methodical scanning – searching for targets based on intelligence common sense.
– Stakeout – awaiting an event in a certain point.
– House call – alternating between several sites in order to search for or retain a target (using the PAYLOAD to handle more than one target simultaneously).
– Observing a small area in an attempt to find a lead so as to proceed with the mission.
The logic of the scan, its rate and real-time camera features are determined by the interpreter to ensure reliability, agility and efficiency in performing the task.
Scanning and Mission Change
While scanning a road in search for roadblocks, the interpreter detected a car driving wildly and leaving a trail of hot water on the road. The mission was consequently changed to following the car, and the recipient was duly notified. The car drove on and parked in a location known to be hostile. The recipient associated the new information with a recent shooting event. This led to the quick capture of the car and its passengers, who later turned out to be wanted men.
Image Interpretation
Any video image (daytime and nighttime) is made of discrete pixels photographed by the detectors. From an interpretive point of view, this means each camera is limited in terms of its resolution ability at maximal zoom.
It also means that certain details appear and disappear suddenly, and that the photographed image must correspond to the level of details required by the query.
Another factor affecting interpretation is atmospheric conditions. Details look different in different times of the day and in various solar angles, humidity levels, clouding and smoke conditions.
Although the monitor shows the viewer an image seemingly familiar to everyone, the details observed and the sequence of events occurring during the mission are in fact the raw materials with which the interpreter works. Only those who have all the information can provide a reliable and complete intelligence response (the gynecologist parable).
The Gynecologist Parable
When future parents visit the gynecologist for an ultrasound scan, we see a blurred and jerky image, in which they enthusiastically detect details of the embryo’s physical shape: facial contours, fingers, or even genitals.
Surprisingly, at the very same time, the skilled doctor or technician interprets the image to assess the regularity of cardiovascular valves, the proportionality of the limbs, and dozens of other medical examinations based on a mandatory protocol.
Moral
Just as the ultrasound is a visual representation of sound waves, the video image of intelligence cameras is a representation of visible light or thermal imagery of an area.
Usually, objects of interest will conceal their activity by camouflage and deceit, and we will be looking for partial clues and leads to obtain the information. The only way to obtain the required information professionally and reliably is to employ a trained professional.
Distributing the Information
Real-time work is usually carried out vis-à-vis a single, known recipient, to whom the information is transmitted exclusively. Sometime, information of interest to other parties is detected, and the UAV crew will report this information to the relevant recipient according to its priority and urgency (although it may not correspond to any particular intelligence query).
Distributing the Information: Agility
When collecting information for a brigade-sized force, the interpreter detected a surface to air fire directed against helicopter gunships engaged in another mission in the area. This information was immediately reported to the Air Force.
On a different mission, in passing between two areas, the interpreter, out of sheer curiosity, looked at an area on which he worked the day before and detected activity known to be unusual. A quick telephone call to yesterday’s recipient clarified the significance of this information, and led to the appropriate response.
Transmitting the Image to Recipients
The UAV is highly capable in terms of transmitting real-time images to distant locations. This capability is well-used by the recipients.
Advantages of Recipient Observation
– Understanding the situation on the ground
– Real-time query updates
– Correcting erroneous queries
– Verifying missing details
– In the case of a skilled recipient, consultation regarding interpretation
Disadvantages of Recipient Observation
– Partial query definitions and the attempt to complete it by self-interpretation
– Image interpretation by unskilled individuals
– The tendency to mange the camera instead of managing the intelligence query (“zoom in, zoom out” instead of “look for a ‘hot’ vehicle in the compound”)
– Undermining operational trust vis-à-vis the mission crew, due to disagreements over image interpretation. For example, a recipient who missed an event reported by an interpreter may disbelieve the information (because he “didn’t see it”) and ignore further reports.
Recipient Interpretation
During an operation to arrest a dangerous wanted man, surveillance was conducted, complete with image transmission to command center. The wanted man dallied with several people for a few minutes and then got into a vehicle which started driving to the area where the mission could be completed. The UAV interpreter identified the target with certainty, according to unique characteristics. The commander, however, cancelled the mission at this point since, quote, “he himself did not identify the target entering the vehicle”.
Conclusion
Interpreting a FMV images is a visint profession. This area of expertise has developed rapidly in recent years, both in terms of technology and in terms of doctrine. Today, it is of crucial operational significance.
Real-time UAV interpretation is a central expertise within the broader realm of FMV interpretation.
In order to make the most of our operational capabilities, we must:
– Select appropriate candidates, in terms of their personal skills and background
– Train professional UAV teams in wide-ranging knowledge areas – technology, intelligence, aviation and operations
– Train recipients so as to make the most of such an advanced visual technology.
Looking to the future, it seems that as technologies become more sophisticated, and with the introduction of computerized interpretation aids and advanced sensors with better ranges and resolution capabilities, the knowledge and training requirements of UAV operators become ever clearer.
The establishment of the UAV School is a positive step in the right direction, but it is not enough. We must keep developing the profession and adapt it to the needs of the foreseeable future.
Acknowledgements
I would like to thank the development team of the FMV interpretation course, and especially Ms. Sigal Kahanovich for their insights.
[*] Classic AP interpretation = Hardcopy photography on a cellulite film, interpreted over table designed especially for that purpose.