Yottabyte Research Cloud able to accept HIPAA-aligned data

By | General Interest, HPC, News

Advanced Research Computing – Technology Services (ARC-TS) is pleased to announce that the Yottabyte Research Cloud (YBRC) computing platform is now HIPAA-compliant. This means that YBRC and its associated services can accept restricted data, enabling secure data analysis on Windows and Linux virtual desktops as well as secure hosting of databases and data ingestion.

The new capability ensures the security of restricted data through the creation of firewalled network enclaves, allowing HIPAA-aligned data to be analyzed safely and securely in YBRC’s flexible, robust and scalable environment.   Within each network enclave, researchers have access to Windows and Linux virtual desktops that can contain any software required for their analysis pipeline.

This capability also extends to our database and ingestion services:

  • Structured databases:  MySQL/MariaDB, and PostgreSQL.
  • Unstructured databases: Cassandra, MongoDB, InfluxDB, Grafana, and ElasticSearch.
  • Data ingestion: Redis, Kafka, RabbitMQ.
  • Data processing: Apache Flink, Apache Storm, Node.js and Apache NiFi.
  • Other data services are available upon request.

YBRC is supported by U-M’s Data Science Initiative launched in 2015. YBRC was created through a partnership between Yottabyte and ARC-TS announced last fall.

These tools are offered to all researchers at the University of Michigan free of charge, provided that certain usage restrictions are not exceeded. Large-scale users who outgrow the no-cost allotment may purchase additional YBRC resources. All interested parties should contact hpc-support@umich.edu.

ARC Director Sharon Broude Geva re-elected vice-chair of Coalition for Academic Scientific Computing

By | General Interest, News

Sharon Broude Geva, the Director of Advanced Research Computing at the University of Michigan, has been re-elected vice-chair of the Coalition for Academic Scientific Computation (CASC).

Founded in 1989, CASC advocates for the use of advanced computing technology to accelerate scientific discovery for national competitiveness, global security, and economic success. The organization’s members represent 84 institutions of higher education and national labs.

The vice-chair position is one of four elected CASC executive officers. The officers work closely as a team with the director of CASC. The vice-chair also leads CASC meeting program committees, is responsible for recruitment of new members, substitutes for the chair in his or her absences, and assists with moderating CASC meetings.

Geva served as CASC secretary in 2015 and 2016, and one term as vice-chair in 2017. Her next term as vice-chair is effective for the 2018 calendar year.

The other executive officers for 2017 are are Rajendra Bose, Chair, Columbia University; Neil Bright, Secretary, Georgia Institute of Technology; and Andrew Sherman, Treasurer, Yale University. Curt Hillegas of Princeton University is immediate past chair.

The 2018 CASC brochure is available online.

U-M wraps up successful SC17 conference

By | General Interest, Happenings, HPC, News

Several University of Michigan researchers and professional IT staff attended the Supercomputing 17 (SC17) conference in Denver from Nov. 12-17, participating in a number of different ways, including demonstrations, presentations and tutorials.

U-M participation included:

  • Matt McLean, a Big Data systems administrator with ARC-TS, served as a panelist at a session titled “The ARM Software Ecosystem: Are We There Yet?” (Slides)
  • Jeff Sica, a research database administrator with ARC-TS, helped lead a Birds of a Feather session titled “Containers in HPC.” (Slides)
  • Quentin Stout (EECS) and Christiane Jablonowski (CLASP) taught the “Parallel Computing 101” tutorial.
  • Shawn McKee, U-M Department of Physics, and OSiRIS Principal Investigator, demonstrated Object Storage and Caching for Science (network topology diagrams)
  • Eric Boyd, Director of Research Networks, presented on Research Networking at the University of Michigan at the U-M exhibit booth.
  • Simon Adorf, Ph.D. Candidate, Chemical Engineering Department, U-M, presented on Simple Data and Workflow Management with Signac and GPU-Accelerated Predictive Material Design at the U-M exhibit booth.
  • ARC sponsored a networking and career networking reception put on by Women in HPC. ARC Director Sharon Broude Geva spoke at the event.
  • Amy Liebowitz, a network architect at ITS, worked on SCINet, a high-capacity network created every year for the conference. Liebowitz was on the routing team, which is responsible for installing, configuring and supporting the high performance conference network. The Routing Team also coordinated external connectivity with commodity Internet and R&E WAN service providers.

New Data Science Course – Winter 2018

By | Educational, News

Computational Data Science
(EECS 598 / BIOINF 505)

A new graduate course that provides an in-depth introduction to computational methods in data science for identifying, fitting, extracting and making sense of patterns in large data sets is now enrolling students for Winter 2018.

Lectures will typically begin with an introduction of a core data science method, followed by the student programming the method computationally with a computer assisting the student by certifying when the program is correct, interleaved with ‘just-in-time’ theory that will expose the student to the mathematics that underpin the methodology. Once the method has been correctly implemented, the students will be given a real world example or ‘success story’ to work with that illustrates when the algorithm ‘works’ as expected, followed by an instructor guided computational exploration of the various subtleties of the algorithm and its weakness.

A full course description, prerequisites and schedule are available.

Please share this announcement with students who might be interested.

U-M students make strong showing at Michigan Datathon

By | Data, Educational, Events, General Interest, Happenings, News

University of Michigan students won first and third places in the Michigan Datathon held Nov. 4, 2017 in the Michigan Union and hosted by Citadel LLC, Correlation One, and the U-M Statistics Department.

1st-place winning team from the University of Michigan:

Ruofei (Brad) Zhao, Statistics Ph.D. student

Zheng Gao, Statistics Ph.D. student

You Wu, Master’s in Applied Statistics student

Kevin Zheng, Sophomore, Computer Science


2nd-place team:

Zi Yi, Statistics Master’s student, University of Chicago

Tian Gu, Biostatistics Ph.D. student, University of Michigan

Shuo Zhang, Statistics Master’s student, University of Chicago

Shiyang Lu, Robotics & Naval Architecture and Marine Engineering Master’s student, University of Michigan


3rd-place team from the University of Michigan:

Hanbo Sun, Master’s in Applied Statistics student

Xinghui Song, Master’s in Applied Statistics student

Tuo Wang, Master’s in Applied Statistics student

Hang Yuan, Master’s in Applied Statistics student


For more, see https://lsa.umich.edu/stats/news-events/all-news/graduatenews/MichiganDatathonWinners0.html

U-M partners with Cavium on Big Data computing platform

By | Feature, General Interest, Happenings, HPC, News

A new partnership between the University of Michigan and Cavium Inc., a San Jose-based provider of semiconductor products, will create a powerful new Big Data computing cluster available to all U-M researchers.

The $3.5 million ThunderX computing cluster will enable U-M researchers to, for example, process massive amounts of data generated by remote sensors in distributed manufacturing environments, or by test fleets of automated and connected vehicles.

The cluster will run the Hortonworks Data Platform providing Spark, Hadoop MapReduce and other tools for large-scale data processing.

“U-M scientists are conducting groundbreaking research in Big Data already, in areas like connected and automated transportation, learning analytics, precision medicine and social science. This partnership with Cavium will accelerate the pace of data-driven research and opening up new avenues of inquiry,” said Eric Michielssen, U-M associate vice president for advanced research computing and the Louise Ganiard Johnson Professor of Engineering in the Department of Electrical Engineering and Computer Science.

“I know from experience that U-M researchers are capable of amazing discoveries. Cavium is honored to help break new ground in Big Data research at one of the top universities in the world,” said Cavium founder and CEO Syed Ali, who received a master of science in electrical engineering from U-M in 1981.

Cavium Inc. is a leading provider of semiconductor products that enable secure and intelligent processing for enterprise, data center, wired and wireless networking. The new U-M system will use dual socket servers powered by Cavium’s ThunderX ARMv8-A workload optimized processors.

The ThunderX product family is Cavium’s 64-bit ARMv8-A server processor for next generation Data Center and Cloud applications, and features high performance custom cores, single and dual socket configurations, high memory bandwidth and large memory capacity.

Alec Gallimore, the Robert J. Vlasic Dean of Engineering at U-M, said the Cavium partnership represents a milestone in the development of the College of Engineering and the university.

“It is clear that the ability to rapidly gain insights into vast amounts of data is key to the next wave of engineering and science breakthroughs. Without a doubt, the Cavium platform will allow our faculty and researchers to harness the power of Big Data, both in the classroom and in their research,” said Gallimore, who is also the Richard F. and Eleanor A. Towner Professor, an Arthur F. Thurnau Professor, and a professor both of aerospace engineering and of applied physics.

Along with applications in fields like manufacturing and transportation, the platform will enable researchers in the social, health and information sciences to more easily mine large, structured and unstructured datasets. This will eventually allow, for example, researchers to discover correlations between health outcomes and disease outbreaks with information derived from socioeconomic, geospatial and environmental data streams.

U-M and Cavium chose to run the cluster on Hortonworks Data Platform, which is based on open source Apache Hadoop. The ThunderX cluster will deliver high performance computer services for the Hadoop analytics and, ultimately, a total of three petabytes of storage space.

“Hortonworks is excited to be a part of forward-leading research at the University of Michigan exploring low-powered, high-performance computing,” said Nadeem Asghar, vice president and global head of technical alliances at Hortonworks. “We see this as a great opportunity to further expand the platform and segment enablement for Hortonworks and the ARM community.”

MDST – NFL Free Agency Value Prediction Competition Kick-Off – Nov. 9, 6pm

By | Data, Data sets, Educational, Events, Happenings, MDSTPosts, MDSTProjects, News

In this competition, student teams at the University of Michigan will use historical free agent data to predict the value of new contracts signed in the 2018 free agency period. These predictions will be evaluated against the actual contracts as they are signed. This competition is organized by the Michigan Data Science Team (MDST), in collaboration with the Baltimore Ravens and the Michigan Sports Analytics Society (MSAS).  Food will be provided. This is an initial kick-off meeting of the competition.


Date, Time

Thursday, November 9 at 6:00 PM EST to Thursday, November 9 at 7:00 PM EST
Add To Google Calendar | iCal/Outlook


Weiser Hall 10th Floor Auditorium
500 Church St, 48104, MI


Michigan Data Science Team



CSCAR provides walk-in support for new Flux users

By | Data, Educational, Flux, General Interest, HPC, News

CSCAR now provides walk-in support during business hours for students, faculty, and staff seeking assistance in getting started with the Flux computing environment.  CSCAR consultants can walk a researcher through the steps of applying for a Flux account, installing and configuring a terminal client, connecting to Flux, basic SSH and Unix command line, and obtaining or accessing allocations.  

In addition to walk-in support, CSCAR has several staff consultants with expertise in advanced and high performance computing who can work with clients on a variety of topics such as installing, optimizing, and profiling code.  

Support via email is also provided via hpc-support@umich.edu.  

CSCAR is located in room 3550 of the Rackham Building (915 E. Washington St.). Walk-in hours are from 9 a.m. – 5 p.m., Monday through Friday, except for noon – 1 p.m. on Tuesdays.

See the CSCAR web site (cscar.research.umich.edu) for more information.

University of Michigan researcher contributes to NASA findings on carbon in the atmosphere showcased in the journal Science

By | General Interest, Happenings, News


High-resolution satellite data from NASA’s Orbiting Carbon Observatory-2 are revealing the subtle ways that carbon links everything on Earth – the ocean, land, atmosphere, terrestrial ecosystems and human activities. Scientists using the first 2 1/2 years of OCO-2 data have published a special collection of five papers today in the journal Science that demonstrates the breadth of this research. In addition to showing how drought and heat in tropical forests affected global carbon dioxide levels during the 2015-16 El Niño, other results from these papers focus on ocean carbon release and absorption, urban emissions and a new way to study photosynthesis. A final paper by OCO-2 Deputy Project Scientist Annmarie Eldering of NASA’s Jet Propulsion Laboratory in Pasadena, California, and colleagues gives an overview of the state of OCO-2 science.

Manish Verma, a Geospatial/Data Science Consultant at the University of Michigan’s Consulting for Statistics, Computing and Analytics Research (CSCAR) unit, contributed as a coauthor to an article on a new way to measure photosynthesis over time and space.

Using data from the OCO-2, Verma’s analysis helped expand the utility of measurements of solar induced fluorescence (SIF), which indicates active photosynthesis in plants. Verma’s work showed that SIF data collected from the OCO-2 satellite provides reliable information on the variability of photosynthesis at a much smaller scale — down to individual ecosystems.

This can, in turn, “lead to more reliable estimates of carbon sources — that is, when, where, why and how carbon is exchanged between land and atmosphere — as well as a deeper understanding of carbon-climate feedbacks,” according to the Science article.

For more, see the NASA press release (https://www.nasa.gov/feature/jpl/new-insights-from-oco-2-showcased-in-science) and the Science article (http://science.sciencemag.org/content/358/6360/eaam5747.full)