John W. Sheppard's Research

Fundamental Research

While much of the work performed in my lab is applied in nature, this work depends on a number of fundamental results. Here we present the main focus areas of this fundamental research.

Continuous Time Bayesian Networks

A relatively recent addition to probabilistic graphical models, CTBNs are "factored" Markov processes that model the evolution of discrete-space systems in continuous time. Our research has focused on extending work in CTBNs by considering improved inference algorithms, augmented representations (compact structures, disjunctive structures, and decision networks), and performing sensitivity analysis. Emerging research is looking at ways to further improve modeling and inference efficiency.

Factored Evolutionary Algorithms

Factored Evolutionary Algorithms (FEA) are a member of the family of cooperative co-evolutionary algorithms where we subdivide high-dimensional optimization problems into subsets of dimensions with overlap. FEA works with any form of stochastic search and has been tested with simulated annealing, genetic algorithms, differential evolution, and particle swarm optimization. We have applied this new model in diverse problem areas including energy-aware routing in sensor networks, abductive inference in Bayesian networks, weight training in deep neural networks, and optimizing a wide variety of multi-modal functions. Future work involves identifying the best "factor architectures" for FEA and determining efficient distributed architectures for implementation.

Transfer Learning

In highly complex and diverse classification and regression problems (e.g., image classification), it is desirable to be able to train a model in one context and reuse the knowledge learned in related contexts. This is called transfer learning. We are investigating methods for learning deep networks for image classification and weather modeling to be transferred based on so-called "lottery tickets" and spatially correlated features.

Explainable Deep Learning

Related to the work in transfer learning, we are seeking new ways to determine what knowledge has been learned by a deep neural network to better improve the reliability of decision making and better determine transferability of knowledge. By applying methods from functional data analysis and spatial statistics, we are developing new tools for identifying, extracting, and reusing such knowledge.

Applied Research

Here we present current and recent funded projects that utilize the results of our fundamental research.

Precision Agriculture

Working with the department of Land Resources and Environmental Science (LRES) at MSU as well as scientists at the University of Illinois at Urbana-Champaign, we are developing methods for designing optimal on-farm experiments for gather data in winter wheat production. We are applying genetic algorithms to develop the on-farm experiments, as well as applying deep learning methods for learning yield/protein maps for guiding the optimization process. This work is part of a larger effort in Data Intensive Agriculture at MSU.

Machine Learning in Hyperspectral/Multispectral Imaging

Through a recent project funded by a corporate sponsor, MSU has been investigating the use of hyperspectral imaging to evaluate produce in grocery stores. The idea is to determine the freshness of the produce, thereby determining its suitability for sale. The related research involves applying a variety of learning methods (including feedforward neural networks, convolutional neural networks, and compressed convolutional networks) for the prediction. Evolutionary methods are also being developed to perform wavelength selection and filter bandwidth identification for low-cost multispectral imagers.

Machine Learning and Topological Data Analysis for Prostate Cancer Diagnosis

Through funding from NSF and NIH, we are investigating alternative feature spaces based on topological data analysis to support learning models for diagnosing the grade of prostate cancer from biopsy and post-prostatectomy slides. The intent is to provide improved diagnosis of Gleason 3+3, 3+4, 4+3, and 4+4 grades with an eye towards improving patient outcomes in diagnosis and treatment. This work is a collaborative effort with researchers at Tulane University, the Tulane Medical Center, and Bozeman Health Deaconess Hospital.

Workplan Optimization in Facility Maintenance

With support from the US Army Construction Engineering Research Lab in Champaign, IL, we are investigating methods for optimizing work plans in the Sustainment Management System (SMS) for the US Department of Defense. This work is currently applying our CTBN-based methods for predictive health in the context of fuel systems (FUELER) and buildings (BUILDER). Other components of SMS expected to benefit from this work include RAILER and WHARFER for rail systems and wharves respectively. Previous work investigated the use of evolutionary algorithms to optimize the workplans. We expect to combine that work with the current CTBN-based work to support overall modeling and optimization.

System-Level Diagnosis and Prognosis

The current focus of Bayesian learning research is in learning Bayesian networks and CTBNs for fault diagnosis and prognosis. Current sponsors are interested in developing networks tied to specific end systems rather than classes of systems with the expectation that these networks better represent the probability distributions underlying their unique operational scenarios. We are also looking at extending these ideas to creating prognostic network through the development of dynamic Bayesian networks and factorial hidden Markov models.

In addition to Bayesian learning, we are actively involved in the development of three families of standards making use of EXPRESS modeling:

  • AI-ESTATE (IEEE 1232): Artificial Intelligence Exchange and Service Tie to All Test Environments
  • SIMICA (IEEE 1636): Software Interface for Maintenance Information Collection and Analysis
  • ATML (IEEE 1671): Automatic Test Markup Language

Of particular interest here is recent work performed to demonstrate the viability of the AI-ESTATE standard for the Department of Defense Automatic Test System Framework. Four "screencasts" of the demonstration are being posted here:

One of the key focus areas for SIMICA is referred to as "diagnostic maturation." Specifically, the semantic models being developed for SIMICA will be used with data mining and analysis algorithms to identify and correct diagnostic deficiencies.

Risk-Based Prognostics and Health Management

Recently, we developed a new model of prognostics and health management (PHM) based on recognizing that emerging hazards are better indicators of the need to perform maintenance than the prediction of underlying faults themselves. With support from the US Navy and NASA, we developed the Continuous time Hazard Analysis and Risk Mitigation (CHARM) system utilizing Continuous Time Decision Networks to provide the foundation for our method. The CHARM) based method incorporates the results of the fundamental work we performed for CTBNs as well as incorporating uncertain evidence, modeling capabilities for disjunctive interaction, and synergistic performance functions. The CHARM-based method is also being applied in the workplan optimization efforts for the US Army.