MODALITY.AI

About the start-up

Modality.AI, Inc. has developed the first automated, clinically validated, multimodal system to assess neurological and psychiatric states, in clinic and remotely.

Modality's conversational AI system monitors neurological and psychiatric conditions: frequently, automatically, accurately, consistently. Researchers and clinicians can now review data in near real-time and monitor treatment response over time, paving the way for more efficient, effective, and less costly clinical trials.

Modality was founded by a team of world-class AI / Machine Learning experts. We collaborate with clinicians and researchers at world-leading institutions such as UCSF, UT Southwestern Medical Center, Massachusetts General Hospital, Purdue, Charité.

Numerous IRB-approved clinical studies have chosen Modality to assess indications including ALS, Parkinson’s, autism, depression, and schizophrenia. Our pipeline includes Alzheimer's Disease, Laryngectomy, Long Covid, and post-stroke impairment.

As a clinically validated and HIPAA-compliant system, participants can now be monitored safely at home, while speech and facial responses are streamed and analyzed. Modality makes it possible to assess patients and participants objectively without installing any special software or apps. Modality works on computers, tablets, and smartphones - just about any device with a browser, webcam, and microphone.

For more information contact Vikram Ramanarayanan or visit Modality.ai.

Project Title

Interpretable Multimodal Technologies for Remote Assessment

Project summary

Modality.AI has a thriving internship program and welcomes opportunities to work with talented graduate interns and Graduate Thesis mentee candidates on furthering the state of the art in interpretable, multimodal, dialog technologies for remote health assessment. The program, among few of its kind operating within startups in industry, is aimed at being mutually beneficial to both Modality’s innovation goals as well as the candidate’s career path, and has resulted in multiple, peer-reviewed publications in international venues.

Candidates are typically involved in R&D in the areas of artificial intelligence (AI), machine learning (ML), digital signal and image processing, specifically applied to the domain of digital health. This includes working on interpretable features and models from data collected using Modality’s multimodal dialog platform, for the assessment and monitoring of neurological and mental health disorders.

RESponsibility

Example tasks include, but are not limited to:

• Writing and maintaining software code to extract novel speech, language, vision and emotion features from multimodal data streams

• Curating and improving data quality

• Investigating novel methods of diagnosis and monitoring of different neurological and mental health conditions (such as ALS, Parkinson’s Disease, Multiple Sclerosis, Alzheimer’s Disease, Schizophrenia and Clinical Depression) via digital tools

• Investigating novel feature extraction methods and machine learning models (including multimodal large language models and deep learning)

• Conducting appropriate machine learning experiments to investigate the accuracy, reliability and generalizability of the proposed metrics

• Publishing results in peer-reviewed conferences and journals. 

About the candidate

Pluses

• Excellent programming skills in Python.

• Graduate-level knowledge/coursework/project work in at least 2 of the following: statistical machine learning (traditional/deep learning methods), probability and statistics, signal/speech/language/image/video processing, computational/cognitive neuroscience)

• Fluent verbal and written communication skills

• Knowledge of AWS

• Proficiency in one or more of mySQL, Javascript, HTML

• Experience with real world system deployments

• Experience with dialog systems or multimodal technologies

• Knowledge/experience with machine learning and deep learning

• Clinical experience (e.g. with physical or mental therapy, speech language pathology, or general medical domain expertise)

• Self driven and motivated to disseminate work internally and externally

• Cross-disciplinary interests (work involves multiple synergies between signal processing, computer engineering, clinical science, linguistics, neuroscience, statistics, UI/UX  and machine learning)