See all apps

Luminosity – a web app for astronomical analyses and visuali

Submitted on April 03, 2013

What problem are you intending to solve?

Easily create interactive scientific visualizations in a web application.

What is the technological approach, or development roadmap?

Modern astrophysics is a domain where emphasis continues to move towards big data. Over the past decade the field has collected data at a rapid rate. The Sloan Digital Sky Survey (SDSS) set a precedent by surveying the sky each night using a robotic telescope. In a few years the Large Synoptic Sky Survey (LSST) will collect higher resolution images with a data generation rate of 3700 gigabytes per night. Storing these data is a challenging task, but so is their consumption.

Astronomers work with a variety of tools to analyze existing data products. Older tools are mostly written in Java, many written more than a decade ago. Current tools are built using high level languages such as Python coupled with scientific libraries. GUI based tools and high level languages are needed to quickly complete common tasks, such as visualization. At the moment there are no web based analysis tools that utilize the latest web technologies.

The wider scientific community has adopted Python, mostly due to established support for scientific libraries. Similarly JavaScript benefits from the huge investment from the technology industry. No more than five years ago, browser vendors began aggressively competing to build faster JavaScript engines. This healthy competition has created an ecosystem for powerful and inherently cross-platform browser based applications. The scientific community can now capitalize from this investment.

The astronomy community will benefit from web applications that access, analyze, and visualize data using the latest HTML5 APIs, WebGL, and hopefully WebCL. Currently astronomical JavaScript libraries are being developed as a communal resource under the astrojs project. The project is young, but it has already received contributions from the wider community. Current libraries allow developers to read standard astronomical data formats, easily visualize high dynamic range images, and handle various astronomical coordinate systems. Each iteration moves these libraries towards more stable and performant implementations.

The capabilities of a browser are no longer in question. Web applications are capable of replacing dated Java-based toolsets, and provide more intuitive workflows. Using existing astrojs libraries, Luminosity, a web-based application, will be developed. This application will couple major features from two primary astronomical applications, SAOImage DS9 and TOPCAT. SAOImage DS9 is built to visualize images, while TOPCAT handles tabular data. Luminosity will be one streamline application that renders two- and three-dimensional images, and tabular data. Reading astronomical data formats will be done using standard JavaScript practices that incorporates Web Workers for intensive data processing. Two-dimensional images and volumetric renderings will be accomplished with WebGL. Generating tables and plots in a web context are a solved problem; their implementation will be straight-forward by utilizing D3, the widely adopted SVG based visualization library. The benefit of consolidating existing tools as a web application means that development will be iterative, rather than scheduled periodic releases requiring explicit upgrades by the user. Additionally, incorporating advanced features may be done rapidly since user interface development is simpler with HTML and CSS compared to GUI frameworks such as Swing or Tk.

Though this application is domain specific, the development process will utilize HTML5 APIs and WebGL in a unique way that may be useful to other disciplines. Much like astronomical images, medical imagery requires special tools to convey their high dynamic range. Each field may share lessons learned so that future development is facilitated.

How will end users interact with it, and how will they benefit?

This project has potential to impact two communities. Primarily the astronomical community will benefit from 1) a collection of robust JavaScript libraries and 2) a new application for analyses and visualizations. Luminosity will provide a means of generating quick visualizations in a flexible environment. Common tasks will be expedited, so that astronomers can focus on their research questions instead of software issues. Installation of additional software will not be needed, potentially allowing a larger audience to use a professional grade research tool. In addition to benefiting astronomers, this project will establish best practices that could be useful to other disciplines. Development practices and solutions to general problems will be shared through blog articles, encouraged by Mozilla Ignite. The code base will be hosted on GitHub, a transparent medium for sharing code and coding practices.

How will your app leverage the 1Gbps, sliceable and deeply programmable network?

Archives containing astronomical data are spread across the world. Traditionally each archive provides a web interface to interact and request datasets. In some cases these data are placed in cold storage requiring requests to be staged before delivery. With a gigabit network and a movement towards building faster and larger storage systems, astronomers will have quicker access to these datasets. Some members of the astronomical community are advocating for archives to adopt CORS. CORS enabled archives will allow web applications, such as Luminosity, to seamlessly retrieve remote data sources so that requesting and analyzing data are blended into one seamless process.

Further application information

Additional supporting information, materials and resources

Read about project updates - project blog

Take a look at the existing code - project repository

Will your work be beta-ready by the end of the Development Challenge?

The web application will be beta-ready, however further modifications can improve reliability when handling gigabyte size files.

How much effort do you expect this work to take?

Achieving a beta release for Luminosity needs approximately one month of dedicated development. Solid groundwork has already been laid with existing astrojs libraries. Each library has been developed with modularity in mind, so that integration with any web application is seamless.

Do you need help?

Yes. The developer will interact with the wider community to garner feedback about which aspects of Luminosity take precedence. Feedback from user interface and user experience experts would be fruitful if available.

If you can help let them know in the comments below.

Amit Kapadia

Citizen Science developer for the Zooniverse. Usually found in a planetarium or climbing a crag.

and team members

The astrojs project is lead by Amit Kapadia, a developer for the citizen science platform Zooniverse, who is eager for his community to adopt standard web technologies. Development of Luminosity will be done by a single developer with input from an international network of technologically savvy astronomers.

comments powered by Disqus