sighingnow
Joined 11 years ago
Karma
180
Submissions
52
Member since
Oct 9
[email protected]
View on Hacker NewsRecent submissions
Comment · 2 years ago
We have unveiled a new webpage that tells the story of GraphScope, charting its course from inception to its future prospects in the field of graph computing. The page explores how Alibaba Cloud has developed GraphScope from its early stages, overcoming challenges and achieving significant milestones to become a crucial tool in graph data processing. This journey highlights not only technological advancements but also practical lessons in scaling, performance, and adaptability.
Comment · 3 years ago
The pipedream-2bw paper[1] and the zero-offload paper[2] both show that 1-step delayed asynchronous gradient update doesn't affect the convergence (and perplexity) while improve the training efficiency (by fully utilize the bubbles in pipeline parallelism) at a large margin.
However, both the Megatron-LM[3] and the DeepSpeed[4] don't use pipedream-2bw scheduling. Could anyone share me some insights or ideas about why such an efficient scheduling scheme doesn't get popular in the LLM pretraining community? Does it suffer convergence/accuracy issue in practice? Or are there any other concerns that blocking it become the default / most popular pipeline parallelism scheduling?
[1]: https://arxiv.org/abs/2006.09503
[2]: https://arxiv.org/abs/2101.06840
Comment · 3 years ago
Optimizing distributed sampling and feature lookup looks really attractive. It's really challenging to deploy GNN training at an industrial-scale for a large graph.
Will GLT be part of graphscope[1] and replacing the current graphscope-for-learning implementation?
Comment · 3 years ago
On March 30th, the Transaction Processing Performance Council (TPC) released its latest rankings: Tencent Cloud's TDSQL set a new record with a performance of 814 million tpmC and a cost of 1.27 RMB per tpmC. This achievement placed TDSQL at the top spot for both tpmC performance and cost-effectiveness in the world.
Comment · 3 years ago
More language bindings, e.g., Python and Rust, will be added later. Don't hesitate to open issues for feature request if you are struggling with handle various different I/O and its dependency hell trouble and find this library might be helpful in your application!