Generative AI Overview Models Generative AI Research Papers Other Online Resources

NEW: “Using Generative AI for Research, a Quick User’s Guide”. Start from this guide on how Generative AI can be used in multiple aspects of your research. 

Welcome to the MIDAS generative AI resource hub. This page is curated to serve researchers looking to integrate Generative AI into their work.

Generative AI models, capable of creating novel, diverse, and coherent content, are revolutionizing numerous domains. They’ve demonstrated their capabilities in fields as diverse as art, music, chemistry, drug discovery, and many more. This hub is in its initial stage, and will be updated frequently. Given the rapid advance of Generative AI, it is not possible for us to build a comprehensive collection. In addition to Generative AI overviews and a list of the most common models, we will focus on featuring examples of how Generative AI is used in research; specialized, “research-use” Generative AI models, as well as models and studies developed by U-M researchers. Please get in touch if you’d like to have your model or research study using Generative AI included in our collection.

 

(Last Updated: September 1st, 2023)

Generative AI Overview

Generative AI in Plain English

A Brief Introduction to GenAI Kenneth N. Reid, PhD (2023). Michigan Institute for Data Science.

What are Generative AI models? IBM Technology

What are Large Language Models (LLMs)? Google for Developers  

What Is ChatGPT Doing … and Why Does It Work? Stephen Wolfram (2023). Stephen Wolfram Writings.

What is Midjourney?  Deanna McLean (2023). Elegant Themes

Generative AI in Technical Terms

On the opportunities and risks of foundation models. Rishi Bommasani, et al., arXiv preprint arXiv:2108.07258 (2021).

ChatGPT is not all you need. A State of the Art Review of large Generative AI models, Roberto Gozalo-Brizuela and Eduardo C. Garrido-Merch´an. arXiv preprint arXiv (2023)

Improving language understanding by generative pre-training. Alec Radford, et al., (2018).

Natural Language Processing with Transformers, Revised Edition. Lewis Tunstall, Leandro von Werra, Thomas Wolf (2022).

Attention is all you need. Ashish Vaswani, et al., Advances in neural information processing systems 30 (2017).

Fine-tuning language models from human preferences. Daniel M. Ziegler, et al., arXiv preprint  (2019).

Examples of General-Use and Specialized Models

Model Commercial use? Outputs Images Outputs Video Outputs NLP Outputs Audio Multi-Modal Input
UM-GPT: ‘AI bot from the University of Michigan, providing information and academic assistance’ No ☐ ☐ ☑ ☐ ☐
Polyglot: A multilingual model ‘with higher non-English language performance’.  Apache 2.0 License ☐ ☐ ☑ ☐ ☐
Jukebox AI: A generative music model to create raw audio music. No ☐ ☐ ☐ ☑ ☐
DALLE 2: Generates images from textual descriptions. Yes  ☑ ☐ ☐ ☐ ☐
DragGAN AI: An image editing / altering tool. No ☑ ☐ ☐ ☐ ☐
Stable Diffusion: Uses text descriptions to generate images. Yes ☑ ☐ ☐ ☐ ☐
Bard: Large language model trained on text and code. No ☐ ☐ ☑ ☐ ☐
ClimateBert: A Pretrained Language Model for Climate-Related Text Apache 2.0 License ☐ ☐ ☑ ☐ ☐
GPT 3.5: Generates content, translate, and answer questions. Yes ☐ ☐ ☑ ☐ ☐
LaMBDA: Good for open-ended chatbot conversations. No ☐ ☐ ☑ ☐ ☐
LLaMA 2: A large language model (LLM) developed by Meta AI. A successor to LLaMA, it has been trained on a larger dataset of text and code. It has outperformed other LLMs on a number of benchmarks. Yes, with limitations. ☐ ☐ ☑ ☐ ☐
Gen-2 : A multimodal AI system that can generate novel videos with text, images, or video clips.
 Yes. Runway users maintain full ownership of their output videos ☐ ☑ ☐ ☐ ☑
MuseNet AI: Produces MIDI format music, given inputs such as genre, artist, or lyrics. No ☐ ☐ ☑ ☑ ☐
VoxelGPT: Translates natural language prompts into data organization and exploration actions such as filtering, sorting, semantic slicing of data, answering queries about data, and searching. Open source ☐ ☐ ☑ ☐ ☑
Midjourney: Generate images from text prompts. Requires Discord installation. Yes for paying users ☑ ☐ ☐ ☐ ☑
Make-A-Video: Converts text prompts into short videos. No ☐ ☑ ☐ ☐ ☑
GPT 4.0: Expansion from GPT-3.5, with image processing and generation. Yes ☑ ☐ ☑ ☐ ☑
Make-A-Scene: Turns text descriptions and freehand sketches into images. No ☑ ☑ ☐ ☐ ☑
Text-To-Room: Generates room-scale textured 3D meshes from text prompts. MIT License ☐ ☑ ☐ ☐ ☐
CoDeF: Image to image video translation with prompts.  MIT License ☐ ☑ ☐ ☐ ☑

Examples of Research Use of Generative AI

Overview

60 ChatGPT Prompts for Data Science (Tried, tested, and rated). Travis Tang. The Medium (2023)

Scientific Discovery in the Age of Artificial Intelligence. Haochen Wang, et al. Nature (2023) 

“So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. Yogesh K. Dwivedi, et al, International Journal of Information Management (2023)

Scientists’ Perspectives on the Potential for Generative AI in their Fields. Meredith Morris. arXiv (2023)

Generative AI: Perspectives from Stanford HAI. Russ Altman, HAI, Stanford University (2023)

Research using Generative AI

Aerospace

SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model. Sequoia Andrade et al. AIAA AVIATION 2023 Forum (2023)

Biomedical Science

Accelerating drug target inhibitor discovery with a deep generative foundation model. Vijil Chenthamarakshan et al., Science Advances (2023)

Combining generative artificial intelligence and on-chip synthesis for de novo drug design. Francesca Grisoni et al., Science Advances (2021)

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing. Yu Gu, et al. arXiv (2023)

Efficient evolution of human antibodies from general protein language models. Brian Hie, et al., Nature Biotechnology (2023)

Large language models generate functional protein sequences across diverse families. Ali Madani, et al. Nature Biotechnology (2023)

Deep generative molecular design reshapes drug discovery. Xiangxiang Zeng, et al., Cell Reports Medicine (2022)

Chemistry

Emergent autonomous scientific research capabilities of large language models. Daniil Boiko et al. arXiv (2023)

Atomic structure generation from reconstructing structural fingerprints. Victor Fung, et al., Machine Learning: Science and Technology (2022)

14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon. Kevin Jablonka, et al., arXiv (2023)

polyBERT: a chemical language model to enable fully machine-driven ultrafast polymer informatics. Christopher Kuenneth and Rampi Ramprasad (2023)

Designing Chemical Reaction Arrays using phactor and ChatGPT. Babak Mahjour et al. ChemRxiv (2023)

Economics

Language models and cognitive automation for economic research. Anton Korinek. National Bureau of Economic Research (2023)

Geography & Geology

Coupled Adversarial Training for Remote Sensing Image Super-Resolution. Sen Lei et al. IEEE Transactions on Geoscience and Remote Sensing (2020)

On the opportunities and challenges of foundation models for geospatial artificial intelligence. Gengchen Mai et al. arXiv (2023)

Manufacturing & Operations Research

Attention-stacked Generative Adversarial Network (AS-GAN)-empowered Sensor Data Augmentation for Online Monitoring of Manufacturing System. Yuxuan Li et al. arXiv (2023)

Medicine

Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. John W. Ayers et al. JAMA Intern Med (2023)

Medical Image Reconstruction Using Generative Adversarial Network for Alzheimer Disease Assessment with Class-Imbalance Problem, Shengye Hu et al. International Conference on Computer and Communications (ICCC) (2020)

Controllable Medical Image Generation via GAN. Zhihang Ren et al. IS&T Int Symp Electron Imaging. 2021 (2021)

Translating radiology reports into plain language using chatgpt and gpt-4 with prompt learning: Promising results, limitations, and potential. Lyu, Qing, et al.- arXiv preprint (2023)

Fine Perceptive GANs for Brain MR Image Super-Resolution in Wavelet Domain. Senrong You, et al. IEEE Transactions on Neural Networks and Learning Systems. (2023)

Robotics

WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images. Jie Li et al. IEEE Robotics and Automation Letters (2017)

Social Sciences

Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark. Minjie Choi et al. arXiv (2023)

Other Online Resources

Online Integrated Development Environments

Online IDEs, or Integrated Development Environments, are cloud-based platforms that provide tools for writing, debugging, and executing code directly within a web browser, without requiring local software installation. They support multiple programming languages and often offer features like collaborative real-time editing, version control, and deployment facilities. For generative AI, online IDEs are beneficial as they provide accessible and convenient platforms for training models, experimenting with different algorithms, sharing work with other researchers, and easily deploying AI applications, without the need for significant computational resources locally. Here are some IDEs to help with practicing with and exploring Generative AIs.

Amazon Sagemaker by Amazon

Azure Machine Learning by Microsoft

Deepnote by Deepnote

Google Colab by Google

Gradient by Paperspace

Kaggle by Kaggle

Other Curated Generative AI Resource Collections

AwesomeLLM – A curated list of LLMs from a technical and AI-focused perspective.

HuggingFace LLM Leaderboard – Generative AI, and especially LLMs, are evolving rapidly. A curated list of top LLMs on a set of benchmarks can be found here.

Papers with Code – A collection of Generative AI (and non-Generative AI) papers with examples of code, datasets and more. A useful resource for seeing Generative AI applications in research.

Stanford Helm – “Holistic Evaluation of Language Models (HELM) is a living benchmark that aims to improve the transparency of language models.”

StreamLit LLM Leaderboard – Another LLM leaderboard that curates from other sources including HuggingFace.