A New Language for Sparse Data

Have a look at a new approach to dealing with sparse data structures.

Yuanming Hu and his team presented Taichi, a new data-oriented programming language that allows managing complex structures. The core here is an advanced data structure-agnostic interface for writing computation code, providing a number of elementary components with different sparsity properties. You can basically use these components to set up different multi-level sparse data structures.

"This decoupling of data structures from computation makes it easy to experiment with different data structures without changing computation code, and allows users to write computation as if they are working with a dense array, states abstract. "Our compiler then uses the semantics of the data structure and index analysis to automatically optimize for locality, remove redundant operations for coherent accesses, maintain sparsity and memory allocations, and generate efficient parallel and vectorized instructions for CPUs and GPUs." 

The team states that the new approach allows developers to quickly experiment with different data structures, achieving "4.55× higher performance on average, compared to hand-optimized reference implementations."

You can find the full paper here. Also, check out the project on GitHub

Join discussion

Comments 0

    You might also like

    We need your consent

    We use cookies on this website to make your browsing experience better. By using the site you agree to our use of cookies.Learn more

    A New Language for Sparse Data