Tag
This paper examines the limitations of general LLM rankings, including benchmark saturation and commercial incentives, and advocates for task-specific evaluation methods to improve reliability.
AI technology has created numerous data collection jobs in China, with hourly wages around 40-80 RMB, which is seen as a novel impact of AI on the employment market.
Amazon Mechanical Turk, a crowdsourcing platform used for tasks like data labeling and research, is shutting down on September 30.
Amazon is discontinuing its crowdsourced work platform Mechanical Turk, which has been widely used for AI training data, due to declining usage and the rise of more efficient data labeling services.
Figure.AI introduces Index, a global platform for crowdsourcing the largest humanoid robot dataset with 16 million videos, where users can contribute and earn money from everyday tasks.
Bribes.fyi is India's first crowdsourced bribe registry that allows anonymous and permanent reporting of bribes across the country.
Proposes a dual-primal graph VAE architecture that treats ground-truth labels as latent variables to aggregate noisy crowdsourced labels, achieving state-of-the-art results on crowdsourcing benchmarks without needing a separate classifier.
This paper proposes an EM-based algorithm that jointly learns item rewards and worker reliability from pairwise comparisons under a Boltzmann-rational model, using Polya-Gamma latent variables for tractable optimization. Experiments show robustness to spammers and adversarial workers in crowdsourcing and reward learning settings.
A rant about using Vibe Coding to take gigs on Zhubajie Crowdsourcing to achieve financial freedom, sharing a thought-provoking requirement.
The article describes how Ukraine is using AI and crowdsourcing to document Russian war crimes, with the Ukraine War Archive processing millions of files for evidence in future court proceedings.
This paper evaluates three NLP approaches (NER, keyword extraction, topic modelling) for automating keyword extraction in crowdsourced collections, using the Their Finest Hour Online Archive as a case study. It finds extractive models like open-weight ones are preferable for responsible deployment, while generative AI poses accountability risks.
A 21-year-old built a free AI agent for World Cup match predictions, contrasting with Tesla's billions spent on autonomous driving, on a platform where users can publish agents and compete for $10,000 in prizes.
The article revisits the SETI@home distributed computing model as a potential solution to alleviate the demand for new database centers by utilizing idle office and home PCs for processing tasks.
StreetComplete is a mobile app that gamifies contributing to OpenStreetMap by presenting users with simple on-site quests to fill in missing map data, directly updating the map in the user's name.
Amazon will stop accepting new customers for Mechanical Turk on July 30, 2026, putting the crowdsourcing service on life support with no new features planned, while existing customers can continue using it.
Anna's Archive offers a $200,000 bounty for obtaining all book scans from Google Books or similar collections, aiming to liberate scanned books that are only exposed as snippets.
A group of AI researchers launched FLARE-AI, a crowdsourced website for reporting and tracking AI misbehavior such as generating malware or leaking personal data, aiming to centralize accountability and transparency in AI safety.
OutYet.ai is a tool that tracks upcoming AI model releases from major labs, providing status updates, release estimates, and alerts. It crowdsources release guesses and verifies access before marking models as available.
Arena, the AI model leaderboard platform originating from UC Berkeley, has reached $100 million in annualized run-rate revenue eight months after launching its commercial service, highlighting the growing demand for AI model evaluation services.
Google's Android earthquake alert system uses smartphone accelerometers to detect P-waves and sends alerts via internet faster than damaging S-waves, providing users with seconds of warning before strong shaking arrives.