ai-playground

AI POCS: ML, NLP, KAN, Reinforcement Learning, audio, translation, object-detection, GAN, LLM, Vision, Stable Diffusion, Classification, Clustering, XGBoost, NM, RNN, CNN, Pytorch, DJL, DeepLearning4J, ONNX, Keras, Tensorflow, OpenCV, nltk, transformers, transformers.js, sentense-transformers, gradio, chroma, langchain, langchain4j, spring-ai, sklearn, gensim, BERTopic, gymnasium, seaborn, matplotlib, pandas, numpy, scipy, scikit-learn, gradio, chroma, langchain4j, spring-ai, gensim, BERTopic, stable-difussion, anthropic, aws bedrock, cohere, flax, gluonnlp, LM Studio, LocalAI, streamlit, Pinecone. And more... All things AI POCS.

Diego's AI Explained: Making Sense of AI (Essay)

My Essay on AI Explained: Making Sense of AI
https://github.com/diegopacheco/ai-playground/blob/main/AI.Explained.DiegoPacheco.pdf

Diego's Notebooks

Traditional AI

Generative AI

Diego's Hugging Face Demos

1. Diego's Story Telling Multimodel LLM Gen AI https://huggingface.co/spaces/diegopacheco/gen-ai-multimodel-fun

2. Diego's Comics Multimodel LLM Gen AI https://huggingface.co/spaces/diegopacheco/genai-llm-multi-model-comics

3. Diego's LLM Image to Labeled Image https://huggingface.co/spaces/diegopacheco/diego-llm-img-to-lbl

4. Diego's CrawlPT https://huggingface.co/spaces/diegopacheco/diego-CrawlPT?logs=container

5. Diego's DialoGPT Chatbot https://huggingface.co/spaces/diegopacheco/diego-llm-chat-DialoGPT?logs=container

Diego's Top POCs

1. Multi-Model Story Telling AI https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-multi-model-story-telling

2. Transformers VQA https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-dqa-donut-base-finetuned-docvqa

3. Llama2 - 7B Unit Test generation https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-CodeGPT-small-java

4. Transformers - Big Code - Generate and Execute Code https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-bigcode

5. Transformers Object Detection https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-object-detection

6. Transformers Pipeline - GPT-like https://github.com/diegopacheco/ai-playground/blob/main/pocs/sentence-transformers-simple-gpt-like/

7. Sentense Transformers - Image Search by Text https://github.com/diegopacheco/ai-playground/tree/main/pocs/sentence-transformers-image-search

8. Reinforcement Learning Playing Atari Pacman https://github.com/diegopacheco/ai-playground/tree/main/pocs/reinforcement-learning-Pacman

9. Transformers.js - Sentiment Analysis React Web App https://github.com/diegopacheco/ai-playground/tree/main/pocs/react-transformers.js-sentiment-analysis/transformers-react

10. Transformers T5 - Translation https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-T5

11. Tabular Q&A - Transformers https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-table-question-anwsering-tapas

12. Transformers DistilBERT - Text Sumarization https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-distilbart-sumarization

13. Transformers DistilBERT - Q&A https://github.com/diegopacheco/ai-playground/tree/main/pocs/transformers-distilbert-qna

14. OpenCV - CNN - Sentiment on Image https://github.com/diegopacheco/ai-playground/tree/main/pocs/opencv-sentiment-from-image

15. OpenCV template matching from images https://github.com/diegopacheco/ai-playground/tree/main/pocs/opencv-template-matching-images

16. Llama3 - Chatbot and QA 1.5 70B (nvidia) (slow on CPU) https://github.com/diegopacheco/ai-playground/tree/main/pocs/nvidia-llama3-ChatQA-1.5-70B

17. NLTK - Sentiment Analysis https://github.com/diegopacheco/ai-playground/tree/main/pocs/nltk-sentiment-analysis

18. Transformers - Llama 3 - 8B Text Generation https://github.com/diegopacheco/ai-playground/tree/main/pocs/llama-3-fun

19. Langchain4J - OpenSearch / Redis - Document Embeddings and Similarity search (Vector Database) https://github.com/diegopacheco/ai-playground/tree/main/pocs/langchain4j-opensearch https://github.com/diegopacheco/ai-playground/tree/main/pocs/langchain4j-redis

20. Langchain4J - LocalAI - Documment Enbeddings and Similarity search https://github.com/diegopacheco/ai-playground/tree/main/pocs/langchain4j-local-ai-document-embeddings

21. Langchain4J - Chatbot with FALCON_7B https://github.com/diegopacheco/ai-playground/tree/main/pocs/langchain4j-chat-FALCON_7B

22. Keras Speam Classification https://github.com/diegopacheco/ai-playground/tree/main/pocs/keras-classificaiton-spam

23. Customer2Vec - Customer Segmentation, clustering and product recomendation by similarity https://github.com/diegopacheco/ai-playground/tree/main/pocs/customer-2-vec-dumb

24. Phi-3-mini-128k Generate Code in Rust https://github.com/diegopacheco/ai-playground/tree/main/pocs/chat-Phi-3-mini-128k-instruct

25. AWS Bedrock LLAMA3 8B Model - Text Generation https://github.com/diegopacheco/ai-playground/tree/main/pocs/aws-bedrock-llama38B-instruct

26. ONNX - Portable format for ML models - export tool https://github.com/diegopacheco/ai-playground/tree/main/pocs/ONNX-model-export

27. DJL - GP2-Text Generation + BERT QA Inference https://github.com/diegopacheco/ai-playground/tree/main/pocs/DJL-gpt2-qa-bert-and-gen-text

28. DJL - Action Recognition in Image https://github.com/diegopacheco/ai-playground/blob/main/pocs/DJL-action-recognition/

29. DJL - Hand Written Digit Recognition https://github.com/diegopacheco/ai-playground/tree/main/pocs/DJL-Handwritten-Recognition-MLP

30. Spring AI + Onnx - Embeddings https://github.com/diegopacheco/ai-playground/tree/main/pocs/spring-ai-ONNX-transformers-all-MiniLM-L6-v2/project

31. SkLearn Linear Regression - Ridge https://github.com/diegopacheco/ai-playground/tree/main/pocs/sklearn-regression-ridge

32. Stable Diffusion 3 - Image Generation https://github.com/diegopacheco/ai-playground/tree/main/pocs/stable-diffusion-3

What AI can do?

Traditional AI:

1. Classification: Classify data into predefined categories.
2. Regression: Predict continuous values.
3. Clustering: Group similar data points.
4. Decision-making: Make decisions based on rules and logic.
5. Optimization: Find the best solution among options.
6. Natural Language Processing (NLP): Understand and generate human language (chatbots, sentiment analysis).
7. Computer Vision: Interpret and understand visual data (image recognition, object detection).
8. Robotics: Control and interact with physical devices.
9. Expert Systems: Mimic human expertise in specific domains.
10. Predictive Maintenance: Predict equipment failures and schedule maintenance.

Generative AI (GenAI):

1. Text Generation: Create new text, such as articles, stories, or conversations.
2. Image Generation: Create new images, such as photos, artwork, or designs.
3. Music Generation: Compose music, melodies, or sound effects.
4. Video Generation: Create new videos, such as animations or clips.
5. Data Generation: Create synthetic data for training or testing.
6. Style Transfer: Transfer styles between images, music, or text.
7. Image-to-Image Translation: Translate images from one domain to another.
8. Text-to-Image Synthesis: Generate images from text descriptions.
9. Dialogue Generation: Engage in conversation, responding to user input.
10. Creative Writing: Generate creative writing, such as poetry or short stories.

Awesome OSS Libraries

Gynasium: A collection of AI Gym Environments for Reinforcement Learning https://gymnasium.farama.org/

BERTopic is a topic modeling technique that leverages 🤗 transformers and c-TF-IDF to create dense clusters https://github.com/MaartenGr/BERTopic

Gensim: Topic Modeling for Humans, LDA, LSI, HDP, DTM, Word2Vec, FastText, Doc2Vec https://radimrehurek.com/gensim/index.html

SentenseTransformers: Multilingual Sentence Embeddings using BERT / RoBERTa / XLM-RoBERTa & Co. https://sbert.net/

nltk: Natural Language Toolkit, Tokenization, Stemming, Lemmatization, POS Tagging, Named Entity Recognition, etc. https://www.nltk.org/index.html

Keras: High-Level Neural Networks API, Convolutional, Recurrent, Dense, Embedding, etc. https://keras.io/

OpenCV: Open Source Computer Vision Library (Image, Video, Face Detection, Object Detection, Tracking, etc.) https://opencv.org/

PyTorch: ML Framework, Tensors, Datasets, DataLoaders, Models, Optimizers, Loss Functions, etc. https://pytorch.org/

Scikit-Learn: ML Library, Classification, Regression, Clustering, Dimensionality Reduction, Model Selection, Preprocessing, etc. https://scikit-learn.org/stable/

Gradio: Build UIs for your machine learning models https://github.com/gradio-app/gradio

Chroma: the AI-native open-source embedding database https://www.trychroma.com/

Transformers.js: A JavaScript library for running large language models in the browser (on local using ONNX) run Hugging face models in the browser. https://github.com/xenova/transformers.js

LocalAI: LocalAI is a platform that enables you to run large language models on your device (local copy of OpenAI's using Hugging Face's transformers) https://localai.io/

Prompt library https://docs.anthropic.com/claude/prompt-library

ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compiler https://onnx.ai/

LangChain: A Decentralized AI Platform for Language Models https://www.langchain.com/

LangChain4J: A Java SDK for LangChain https://github.com/langchain4j/langchain4j

DL4J: Deeplearning4j is an open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Spark, DL4J is designed to be used in business environments on distributed GPUs and CPUs https://deeplearning4j.konduit.ai/

Deep Java Library (DJL): An open-source, high-level, engine-agnostic Java framework for deep learning built by AWS https://docs.djl.ai/index.html

SpringAI: Spring AI is a Spring project that aims to provide a simple and consistent way to work with AI and ML libraries in the Spring ecosystem https://spring.io/projects/spring-ai

Papers

AWS Recommended

ReAct: Synergizing Reasoning and Acting in Language Models https://arxiv.org/abs/2210.03629

Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network https://arxiv.org/abs/1808.03314

Attention Is All You Need https://arxiv.org/abs/1808.03314

High-Resolution Image Synthesis with Latent Diffusion Models https://arxiv.org/abs/2112.10752

Tree of Thoughts: Deliberate Problem Solving with Large Language Models https://arxiv.org/abs/2305.10601

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks https://arxiv.org/abs/2005.11401

Other industry-based papers

Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? https://arxiv.org/abs/2405.05904

Chameleon: Mixed-Modal Early-Fusion Foundation Models https://arxiv.org/pdf/2405.09818

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf

Images that Sound: Composing Images and Sounds on a Single Canvas https://ificl.github.io/images-that-sound/ https://arxiv.org/pdf/2405.12221

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5) https://arxiv.org/pdf/1910.10683

Researchers say their new algorithm trained on a database of TV show Friends clips can detect sarcasm 75% of the time. https://www.popsci.com/technology/sarcasm-algorithm/ https://aclanthology.org/P19-1455.pdf

xLSTM: Extended Long Short-Term Memory https://arxiv.org/pdf/2405.04517

Measuring Massive Multitask Language Understanding (MMMLU) https://arxiv.org/pdf/2009.03300

RAGAS: Automated Evaluation of Retrieval Augmented Generation https://arxiv.org/pdf/2309.15217

From LLM to NMT: Advancing Low-Resource Machine Translation with Claude https://arxiv.org/pdf/2404.13813.pdf

Batch Prompting: Efficient Inference with Large Language Model APIs https://arxiv.org/pdf/2301.08721v1.pdf

The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey https://arxiv.org/abs/2404.11584

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone https://arxiv.org/abs/2404.14219

Automated Discovery of Functional Actual Causes in Complex Environments https://arxiv.org/pdf/2404.10883.pdf

Can Large Language Models Infer Causation from Correlation? https://arxiv.org/abs/2306.05836

RewardBench: Evaluating Reward Models for Language Modeling https://arxiv.org/pdf/2403.13787v1.pdf

Simple and Scalable Strategies to Continually Pre-train Large Language Models https://arxiv.org/abs/2403.08763

Longformer: The Long-Document Transformer https://arxiv.org/pdf/2004.05150.pdf

Efficient Estimation of Word Representations in Vector Space (Word2Vec) https://arxiv.org/pdf/1301.3781.pdf

Code Llama: Open Foundation Models for Code https://arxiv.org/pdf/2308.12950.pdf

From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples https://arxiv.org/pdf/2404.07544.pdf

ReFT: Representation Finetuning for Language Models https://arxiv.org/pdf/2404.03592.pdf

OLOMO: Accelerating the Science of Language Models https://arxiv.org/pdf/2402.00838.pdf

FinanceBench: A New Benchmark for Financial Question Answering https://arxiv.org/pdf/2311.11944.pdf

Routerbench: A Benchmark for Multi-LLM Routing System https://arxiv.org/pdf/2403.12031.pdf

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding https://arxiv.org/pdf/1810.04805.pdf

Long-form factuality in large language models https://arxiv.org/pdf/2403.18802.pdf

Is GPT-4 a Good Data Analyst? https://arxiv.org/abs/2305.15038

Chronos: Learning the Language of Time Series https://arxiv.org/abs/2403.07815

Automated Unit Test Improvement using Large Language Models https://arxiv.org/pdf/2402.09171.pdf

Models

Progress Tracker: Open vs. Proprietary LLMs https://huggingface.co/spaces/andrewrreed/closed-vs-open-arena-elo

PaliGemma – Google's Cutting-Edge Open Vision Language Model https://huggingface.co/blog/paligemma

StarCoder2-Instruct: A Large Language Model for Code Generation https://github.com/bigcode-project/starcoder2-self-align

FinGPT: Open-Source Financial Large Language Models https://github.com/AI4Finance-Foundation/FinGPT

LLAMA 2: The Language Model for the Open Web https://huggingface.co/docs/transformers/en/model_doc/llama2

T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer https://huggingface.co/docs/transformers/en/model_doc/t5

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding https://huggingface.co/docs/transformers/en/model_doc/bert

GPT-2: Language Models are Unsupervised Multitask Learners https://huggingface.co/docs/transformers/en/model_doc/gpt2

Posts

Neural Network Zoo https://www.asimovinstitute.org/neural-network-zoo/

Solve puzzles. Improve your pytorch. https://github.com/srush/Tensor-Puzzles

llama3 implementation one matrix multiplication at a time https://github.com/naklecha/llama3-from-scratch

History of the Generative AI https://medium.com/@glegoux/history-of-the-generative-ai-aa1aa7c63f3c

Convolutional Neural Network (CNN): A Complete Guide https://learnopencv.com/understanding-convolutional-neural-networks-cnn

Stable Diffusion 3: Research Paper https://stability.ai/news/stable-diffusion-3-research-paper

OWASP Top 10 for Large Language Model Applications https://owasp.org/www-project-top-10-for-large-language-model-applications/

Videos

Generative AI in a Nutshell https://www.youtube.com/watch?v=2IK3DFHRFfw&ab_channel=HenrikKniberg

Building Production-Ready RAG Applications https://www.youtube.com/watch?v=TRjq7t2Ms5I&ab_channel=AIEngineer

All Learning Algorithms Explained in 14 Minutes https://www.youtube.com/watch?v=BT6Aw6Q75Yg&ab_channel=CinemaGuess

3Blue1Brown: Neural Networks and Deep Learning animated math https://www.3blue1brown.com/topics/neural-networks

Let's build GPT: from scratch, in code, spelled out. https://www.youtube.com/watch?v=kCc8FmEb1nY&ab_channel=AndrejKarpathy

ML on Kubernetes

KServe https://github.com/kserve/kserve

Kubeflow https://www.kubeflow.org/

MLOperator https://github.com/polyaxon/mloperator

Interesting Solutions

The Best LLM on Every Prompt (LLM Routiung) https://unify.ai/

Vectara (RaG as a Service) https://vectara.com/