Skip to main content

In recent years there has been tremendous progress in the development of Artificial Intelligence (AI), Autonomy, and Augmentation (AAA) technologies, but less success is translating concepts into operationalizing capabilities. As a trusted agent to the DoD and IC, ARLIS applies a human-centered approach to incorporate into operational workflows AAA technologies that are trusted, reliable, and safe.

ARLIS supports operationalizing AAA technologies on multiple fronts: trusted test and evaluation, human-system integration, and foundational AAA research and development (R&D) in support of the first two categories.

  • Trusted support work at the systems engineering level has included test and evaluation of integrated systems against curated data and scenarios and evaluate the security and robustness of AAA technologies (e.g., via red teaming via adversarial methods).
  • In our mission- and human-centered analysis and evaluation of technology, we work with government sponsors to perform workflow analysis and mission modeling, design and evaluate human-machine teams, and conduct user-centric operational test and evaluation.
  • ARLIS is performing foundational AAA R&D in pursuit of operationalization.  This includes formal methods and simulation-based verification to support test, evaluation, verification, and validation (TEV&V) for artificial intelligence and autonomy and peripheral nerve stimulation and blended reality displays for prototyping and demonstrating next-generation cognitive augmentation.
br

At the request of the Office of the Director of National Intelligence (ODNI) and in partnership with Carnegie Mellon University's Software Engineering Institute, ARLIS is building an R&D roadmap for AI system engineering. This process facilitates research across academia focused on developing tools and methodologies for a robust process, which includes formal methods for AI-based system specification and verification, simulation-based test and evaluation, man-machine teaming, and human-computer interaction.

The HCIL will be used for tasks ranging from incremental usability evaluation early in development to final test and evaluation feedback. By ensuring that the technologies that operators and analysts will be using in the future are optimized at the sociotechnical system level, ARLIS testbeds and labs will increase effectiveness for new, emerging AAA technologies and improve capabilities across the IC and DoD.

robo

Putting Human-System Integration to the Test

ARLIS is currently developing a testbed to evaluate AAA technologies at the mission-level for the intelligence community --- to maximize the overall quality of support the tools provide, identify how they might be best employed, and anticipate how they might be harmful.   This facility will enable ARLIS to test a wide range of scenarios in technical settings and operational user workflows.  As part of this effort, ARLIS is building out a new Human-Computer Interaction Laboratory (HCIL),) within its secure facility, which will accommodate usability and accessibility evaluation for a wide range of AAA technologies, including information visualization, extended reality, and artificial intelligence solutions.

 

Physiological Factors of Human Performance

The field of human performance is in a moment of rapid expansion. Technological advancements have made it possible to study, predict, and improve human performance with a precision previously unimagined. Wearable and increasingly sophisticated physiological sensors provide rich and nuanced data about individual traits and states. The availability and quality of physiological data make it possible to examine performance factors outside of the lab and, thus, accelerate the transition from basic to applied research.

Hand in hand with collecting the need to collect large amounts of continuous and high-resolution data is the ability to analyze it. Recent advances in data analytics (e.g., AI, and machine learning) have enabled researchers to model and understand physiological data in unprecedented ways, increasing the capability for detecting patterns and profiles can better understand the neural, cognitive, and biological factors that affect performance. Finally, the ability to influence and improve performance is at our fingertips. Advances in noninvasive peripheral nerve stimulation techniques have spurred a surge in interest in these approaches to enhancing cognitive and physical performance. ARLIS is a leader in the field of human performance, with its capabilities in the following:

 

      Psychometric assessment, including the development of aptitude batteries of aptitude tests to improve selection in training and workforce contexts (e.g., cybersecurity and, language training);

      Neuroimaging and psychophysiological index development, including improved understanding of how cognitive load and affective states influence performance;

      Physical activity measures, including the relationship between physical activity/ability indices (e.g., accelerometer, gait, cardiovascular health, and lifestyle information) and performance;

      Interventions to enhance performance, including the development and testing of noninvasive peripheral nerve stimulation (PNS) techniques to improve learning and performance;

      Speech and national language processing, including to extract information about affective and cognitive states based on linguistic output; and

      Team science expertise, both in terms of optimizing trust and usability in human-machine teaming contexts and psychology-based approaches to optimize human team performance.

ARLIS advances in noninvasive peripheral nerve stimulation techniques have spurred a surge of interest in these approaches to enhancing cognitive and physical performance, including in areas as disparate as reducing cognitive bias in analysts and reducing PTSD effects in service members.

Current Ventures

A Trusted Partner to Enable AI Innovation

The ARLIS team is a trusted evaluation partner on the DARPA’s Hierarchical Identify Verify Exploit (HIVE) and Software-Defined Hardware (SDH) programs. Both of these programs are part of DARPA’s Electronics Resurgence Initiative, and which in this capacity is developing and running experiments for the AI/ML-accelerator technologies being developed under the programs. These technologies include cutting-edge, experimental hardware designed to support hybrid sparse-dense machine learning workflows and large-scale graph analytics, as well as algorithms to support a variety of mission-critical tasks. ARLIS is playing a key role in every step of the testing and evaluation life cycle, from problem identification and dataset selection/generation to performance analysis, including scalable, reproducible benchmarking for trade-off aware comparative analysis.

i

Avatar Embodiment: Team members become virtually collocated avatars and interact with each other, and shared objects such as the military symbologies.

AI and VR to Augment Mission Planning

For the past two years, ARLIS worked with a team from the Army Futures Command C5ISR Center to develop a prototype distributed collaboration tool based on virtual reality (VR) technology.  The prototype enables physically distributed commanders to plan and rehearse a mission around a virtual "sand table" as if they were physically co-located.  The distributed planners collaborate around a virtual representation of the physical terrain, building plans using standard military symbology, and then rehearsing the scenarios through simulation, over-constrained communication channels

Roomscape with Tabletop

Teammates instantiate around a table that shows targeted landscapes and has a menu to create objects, see a 2D map, save current states, and more

AI and VR to Augment Mission Planning

In 2019 and 2020, ARLIS worked with a team from the Army Futures Command C5ISR Center to develop a prototype distributed collaboration tool based on virtual reality (VR) technology.  The prototype enables physically distributed commanders to plan and rehearse a mission around a virtual "sand table" as if they were physically co-located.  The distributed planners collaborate around a virtual representation of the physical terrain, building plans using standard military symbology, and then rehearsing the scenarios through simulation, over- constrained communication channels.

A Trusted Partner to Enable AI Innovation

The ARLIS team is a trusted evaluation partner on the DARPA's Hierarchical Identify Verify Exploit (HIVE) and Software-Defined Hardware (SDH) programs both. Both of these programs are part of DARPA's Electronics Resurgence Initiative —, and in this capacity, is developing and running experiments for the AI/ML-accelerator technologies which are being developed under the programs.  These technologies include cutting-edge, experimental hardware designed to support hybrid sparse-dense machine learning workflows and large-scale graph analytics, as well as algorithms to support a variety of mission-critical tasks on operational datasets..  ARLIS has played a crucial role in every step of the testing and evaluation life cycle, from problem identification and dataset selection/generation to performance analysis to scalable, reproducible benchmarking for the purpose of trade-off aware comparative analysis.