Big data is a big deal these days, and it’s our connected systems that’s driving demand around number-crunching and analysis in this department.  Social networks alone have driven the desire to connect relationships across the web, adding another sticking point for massive graph processing, which powers notable engines like Google Search’s Page Rank.  Responding to the need for layering in more relational analysis amongst data points, systems like  Pregel emerged from Page Rank’s two-dimensional MapReduce.  And just as Hadoop was spawned from a similar pet project at Yahoo, GoldenOrb hopes to extend the Pregel model for massive, scalable graph processing in the cloud.

Head by Austin-based big data analytics software maker Ravel, GodenOrb is an open source initiative that deploys large-scale graph analysis at an affordable level.  Tailor-made to crunch terabytes of data, GoldenOrb’s angle is that it maintains the relationships between the data.  GoldenOrb is based on Apache Hadoop, picking up a torch that’s inspired a wealth of open source projects for processing data.  Much of Web 2.0 was built on it, scraping context from tweets and planting ideas of real-time capabilities in the minds of developers.

Web 2.0 and other trends in research and analysis have taught us that linear data analysis, such as that found in MapReduce, isn’t enough to solve all the issues around how  data can in fact drive decisions.  Transforming data from the MapReduce format adds a great deal of time and energy to analysis, and as GoldenOrb Lead Architect and President Zach Richardson tells me, “one of the reasons we started this is because we have so many more problems to solve.”

Richardson goes on to list a few scenarios in which GoldenOrb may be useful, starting with social calculations.  But Ravel’s new project is good for more than just determining the stability of a clique within a social network.  It’s large-scale graph analysis can be applied to research in medical and scientific disciplines like epidemiology, looking at the spread of disease across a nation.

The realization that so many more vertexes can be compared against each other with large-scale graph analysis opens the door to garnering answers where they wouldn’t have easily been found.  Scaling up is a matter for GoldenOrb’s sophisticated algorithm, and isn’t limited to the number of machines that are available to process data points.  Running these algorithms in a time-efficient manner is what brings value to any ecosystem developed around GoldenOrb, though there are a few other factors important to Ravel’s latest initiative.

“We wanted to create something that would fit into the existing framework,” Richardson says.  “We believe that an open development model is the only way to foster solutions to complex data problems.”

For GoldenOrb, that means finding a way to fit into the current big data ecosystem, which is dominated by solutions including Hadoop and Cassandra.  With ready points of integration and a penchant for scalability, GoldenOrb is hoping to gain clients and partners through ease-of-use, appealing to enterprise solutions and other developer projects that can glean their own value from this type of open source platform.