hzj5790
Joined 3 years ago
Karma
14
Submissions
8
Member since
Mar 12
Recent submissions
Comment · 3 years ago
Abstract: Large language models (LLMs) have achieved remarkable success in NLP and multimodal tasks. Despite these successes, their development faces two main challenges: (i) high computational cost; and (ii) difficulty in conducting fair and objective evaluations. LLMs are prohibitively expensive, making it feasible for only a few major players to undertake their training, thereby constraining both research and application opportunities. This underscores the importance of cost-effective LLM training. In this paper, we utilize a growth strategy to significantly reduce LLM training cost. We demonstrate that an LLM with 101B parameters and 0.31TB tokens can be trained on a $100K budget. We also adopt a systematic evaluation paradigm for the IQ evaluation of LLMs, in complement to existing evaluations that focus more on knowledge-oriented abilities. We introduce our benchmark including evaluations on important aspects of intelligence including symbolic mapping, rule understanding, pattern mining, and anti-interference. Such evaluations minimize the potential impact of memorization. Experimental results show that our model FLM-101B, trained with a budget of $100K, achieves comparable performance to powerful and well-known models, e.g., GPT-3 and GLM-130B, especially in the IQ benchmark evaluations with contexts unseen in training data. The checkpoint of FLM-101B will be open-sourced at https://huggingface.co/CofeAI/FLM-101B.
Comment · 3 years ago
From the Article:
"One of Walmart’s latest offerings at its SuperCenters isn’t a hot new toy, snack flavor or sundress. It’s advertising.
Shoppers will soon see more third-party ads on screens in Walmart self-checkout lanes and TV aisles; hear spots over the store’s radio; and be able to sample items at demo stations.
Walmart’s push into advertising resembles similar moves by retailers like Kroger, which struck a deal to bring digital smart screens to cooler aisles in hundreds of its stores, and Target, which began testing in-store demos and giveaways, including a recent “Barbie” branded event with Mattel that took place at about 200 stores.
For Walmart, selling ad space to its wealth of existing partners is another way to capitalize on the company’s huge reach and to expand into higher-margin businesses. The discounter has nearly 4,700 stores across the U.S., with roughly 90% of Americans living within 10 miles of a Walmart store.
In the U.S., about 139 million customers visit Walmart stores and its website or app each week.
“When you think about our store, our store footprint and the the percentage of Americans that we reach through our stores, we can deliver Super Bowl-sized audiences every week,” said Ryan Mayward, senior vice president of retail media sales for Walmart Connect, the retailer’s advertising business.
The company plans to ramp up in-store ads using its approximately 170,000 digital screens across its locations as well as 30-second radio spots that will be available to suppliers later this year and can target a specific store or region.
And it’s hoping at least one of the new advertising initiatives will be easy to digest: Free samples in stores on the weekends.
Walmart plans to sell the sampling stations to advertisers and bundle them with other ad formats that can run at the same time to make for a fuller campaign. QR codes at the demo tables will pull up online shopping options, meal ideas or seasonal information.
It tried out the new in-house approach of selling sampling stations in Dallas-Fort Worth and plans to offer the option in over 1,000 stores across the country by end of January."
Comment · 3 years ago
From the PR:
"From an economical and ecological perspective the current "Non-commercial bespoke" model license is sub-optimal and should be changed to a truly liberal open-source license like for example Apache 2.0.
In the current state Meta published the whole replication recipe open-source (GPL v3) but asks other entities to spend a lot of energy (potentially releasing massive amounts of CO2 into the atmosphere) to replicate and release a truly open-source version of LLaMA. Given the fact that LLaMA model weights are currently already available for download at many different places this is from an ecological perspective a preposterous management decision and in my personal opinion not well aligned with the overall ecological ambitions of Meta.
If you say "open" (as in the LLaMA paper) and you want to get the bonus credibility that comes with it .. please do it fully and not half-hearted as done currently."