Difference between revisions of "Free Software Directory talk:Artificial Intelligence Team"

From Free Software Directory
Jump to: navigation, search
m (Moving March '23 copyright guidance to main, creating benchmarks and architectures category to clean up main)
m (Text: adding huggingface uncensored leaderboard + alphabetizing list)
Line 38: Line 38:
  
 
==== Text ====
 
==== Text ====
* [https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard HuggingFace's Open LLM leaderboard]
 
  
* [https://tatsu-lab.github.io/alpaca_eval/ AlpacaEval Leaderboard]
+
* [https://tatsu-lab.github.io/alpaca_eval/ AlpacaEval]
  
* [https://chat.lmsys.org/?leaderboard Large Model Systems Organization Leaderboard]
+
* [https://rentry.org/ALLMRR Another LLM Roleplay Rankings]
  
* [https://github.com/FranxYao/chain-of-thought-hub#results Yao Fu's (FranxYao) chain-of-thought hub]
+
* [https://ayumi.m8geil.de/ayumi_bench_v3_results.html Ayumi Benchmark v3]
 +
 
 +
* [https://lifearchitect.ai/models-table Googlesheet of models, AI labs, datasets, and various other ML info by Alan Thompson]
  
* [https://huggingface.co/spaces/mike-ravkine/can-ai-code-results Mike Ravkine's CanAiCode Leaderboard 🏆]
+
* [https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard HuggingFace BigCode Models]
  
* [https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard BigCode Models Leaderboard]
+
* [https://huggingface.co/spaces/mike-ravkine/can-ai-code-results HuggingFace Mike Ravkine's CanAiCode]
  
* [https://ayumi.m8geil.de/ayumi_bench_v3_results.html Ayumi Benchmark v3]
+
* [https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard HuggingFace Open LLM]
  
* [https://rentry.org/ALLMRR Another LLM Roleplay Rankings]
+
* [https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard HuggingFace Uncensored General Intelligence]
  
* [https://lifearchitect.ai/models-table Googlesheet of models, AI labs, datasets, and various other ML info by Alan Thompson]
+
* [https://chat.lmsys.org/?leaderboard Large Model Systems Organization]
  
 
* [https://rank.opencompass.org.cn/home OpenCompass (China)]
 
* [https://rank.opencompass.org.cn/home OpenCompass (China)]
 +
 +
* [https://github.com/FranxYao/chain-of-thought-hub#results Yao Fu's (FranxYao) chain-of-thought hub]
  
 
==== Voice ====
 
==== Voice ====

Revision as of 19:37, 18 March 2024

Free software replacements that are missing

  • AI Research Assistant
    • https://elicit.org/ - Elicit uses language models to help you automate research workflows, like parts of literature review.
  • Voice to instrument: Tone Transfer-like
  • Identification
    • Photo
    • Audio
      • Shazam: Shazam is an application that can identify music, movies, advertising, and television shows, based on a short sample played and using the microphone on the device.
      • A Shazam-like software that is identifying genres instead of songs.
      • A free app that functions like midomi.com -- "You can find songs with midomi and your own voice. Forgot the name of a song? Heard a bit of one on the radio? All you need is your computer's microphone."
  • http://design.rxnfinder.org/addictedchem/prediction/

Potential Freedom issues

  • Dependencies need to be checked.
  • Verify whether a workflow requires non-free GPU or if CPU can be used.
  • The training data often contains non-free licensed material.
    • According to current copyright laws, this does not impact the license of the model or the output of the model. According to current copyright laws, the output is public domain. Mmcmahon (talk) 11:48, 2 May 2023 (EDT)

Model licenses

There appears to be a swath of custom model licenses being used independent of the more standardized software licenses used to interact with models. This presents a conflict as to what license is deemed applicable to the files contained in any repo.

Reddit - Security PSA: huggingface models are code. not just data.

This video (starting at 16:50) illustrates a good argument that model checkpoints may not fall under copyright protection so traditional software licenses that depend on copyright law would be invalid. The video does illustrate that contract law may try to be used it place of copyright. I would advise not using YouTube directly and instead using yt-dl or Invidious.

Worth noting: Open LLaMA (out-of-date) is an example of removing the issue of model cards for text generation.

StabilityAI keeps (Nov '22) updating (July '23) its licenses (Nov '23), so I've removed them from the main page until they settle down.

GLIGEN and GLIGEN GUI is quite neat, but states strict model terms and conditions associated with using it.

Testing model viability

Tools are needed to assess the pros/cons of each model.

Leaderboards

Due to the issue of merely training a model to become good at whatever tests are on a leaderboard, multiple leaderboards are preferential (hence not putting HuggingFace on the main page). A more comprehensive evaluation would be a meta-analysis of existing leaderboards.

Text

Voice

Benchmarks

Architectures

  • Hyena

Ordinal value scales could exist for

Source of model training data

  • amount of data
  • date range (e.g. distinguishing old science from new science for smaller scale models)
  • level of censorship (important to make personal+research use distinct from business use)

Problem solving

  • math
  • creative problem solving (there exists methodology for testing this in humans)

"The present findings suggest that the current state of AI language models demonstrate higher creative potential than human respondents." Nature (Feb 10, '24)

General trends

  • Larger models are more prone to human superstition[1], but also generate more human-like readability.
  • Quantization (a la GPT-Q) allows consumer hardware to run large models.

Stable Diffusion

Stable Diffusion model files (.ckpt) are released under a non-free license.

Here's the stable diffusion beginning point: https://huggingface.co/CompVis/stable-diffusion-v1-4 https://huggingface.co/spaces/CompVis/stable-diffusion-license

stable-diffusion-webui

Large Language Models

Censorship issues

A guide to decensoring models; I would exercise caution, as it stands to reason an inherently uncensored model would perform better than needing the legwork of decensoring one (and then making mistakes + missing some of the censorship)

Legacy notable projects

Project Credit License Description
DALL-E Mini borisdayma (Boris Dayma) Apache 2.0 Generate images from a text prompt
neural-style anishathalye GPLv3 An implementation of neural style in TensorFlow

External links



Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the page “GNU Free Documentation License”.

The copyright and license notices on this page only apply to the text on this page. Any software or copyright-licenses or other similar notices described in this text has its own copyright notice and license, which can usually be found in the distribution or license text itself.