mentatseb

Yup, it's important that people understand the role of visualization in the complete data chain.

Hi, I'm Gephi+Linkurious co-founder. I've found visualizing large graphs pretty useless beyond the "I see meatballs!" effect and my opinion, after a decade in the field, is that it's the wrong problem for data analytics.

Much more interesting information is discovered during the process of dynamically building a visualization that is focused on user questions. I see with Linkurious that investigators usually need to visualize less than 1000 edges of a 1M+ edges graph to get answers.

The conclusion is misleading due to 2 wrong assumptions:

1. The population is heterogeneous: interviews test different skills. All interviews don't test the same set of skills, which is mandatory to compare interview scores because scores are aggregates of these skill tests. Different job opportunities means different skills to test, so it seems reasonable to assume that people evaluation vary for different job opportunities, and thus their scores vary for different interviews.

2. The observations are not statistically independent: past interviews may influence future interviews. People may get better at passing interviews or conducting interviews over time. This would impact their score. It would be good to study the evolution of individual scores over time.

While (1) should strongly limit the conclusions of the study, the complete analysis may simply be irrelevant because of (2) if the statistical independence of observations is not demonstrated. Sorry guys but this is Statistics 101 introductory course.

Thanks, I've fixed it.

The biggest dataset used by one of our users is a genetic graph of 240 millions of nodes and edges using a single server. Linkurious will take a few hours to index the complete dataset. From then, the search engine delivers instant results with autocomplete, fuzziness and advanced query options; graph exploration queries take less than a second to complete (sometimes a bit more depending on the web client). We are still working on improving our data indexing strategy to gain performance.

Synerscope has a strong approach to data analysis, and Danny Holten is well-known in the infovis community. I don't think that they provide a search engine thought, you have probably more information on their product.

Thanks! Worth also mentioning Sigma.js, the open source core on which we are building our toolkit.