Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI #2355

Merged
merged 57 commits into from
May 2, 2024
Merged

📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI #2355

merged 57 commits into from
May 2, 2024

Commits on Apr 24, 2024

  1. Basic implementation of ChatGPT conversation import

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    231e70a View commit details
    Browse the repository at this point in the history

  2. remove debug code

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    f3dceec View commit details
    Browse the repository at this point in the history

  3. Handle citations

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    5a971b4 View commit details
    Browse the repository at this point in the history

  4. Fix updatedAt in import

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    3b7b160 View commit details
    Browse the repository at this point in the history

  5. update default model

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    6df3a8b View commit details
    Browse the repository at this point in the history

  6. Use job scheduler to handle import requests

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    9ce357d View commit details
    Browse the repository at this point in the history

  7. import job status endpoint

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    02548a1 View commit details
    Browse the repository at this point in the history

  8. Add wrapper around Agenda

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    68eacf6 View commit details
    Browse the repository at this point in the history

  9. Rate limits for import endpoint

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    5e94867 View commit details
    Browse the repository at this point in the history

  10. rename import api path

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    f067c90 View commit details
    Browse the repository at this point in the history

  11. Batch save import to mongo

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    d9f7a30 View commit details
    Browse the repository at this point in the history

  12. Improve naming

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    78c48d9 View commit details
    Browse the repository at this point in the history

  13. Add documenting comments

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    ad9da4e View commit details
    Browse the repository at this point in the history

  14. Test for importers

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    304a419 View commit details
    Browse the repository at this point in the history

  15. Change button for importing conversations

    @jakubmieszczak @danny-avila
    jakubmieszczak authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    97b5ae1 View commit details
    Browse the repository at this point in the history

  16. Frontend changes

    @jakubmieszczak @danny-avila
    jakubmieszczak authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    45a2010 View commit details
    Browse the repository at this point in the history

  17. Import job status endpoint

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    2f3fc19 View commit details
    Browse the repository at this point in the history

  18. Import endpoint response

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    fd5e798 View commit details
    Browse the repository at this point in the history

  19. Add translations to new phrases

    @jakubmieszczak @danny-avila
    jakubmieszczak authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    7a34d41 View commit details
    Browse the repository at this point in the history

  20. Fix conversations refreshing

    @jakubmieszczak @danny-avila
    jakubmieszczak authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    5d30dcc View commit details
    Browse the repository at this point in the history

  21. cleanup unused functions

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    ae26ccc View commit details
    Browse the repository at this point in the history

  22. set timeout for import job status polling

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    ff559c8 View commit details
    Browse the repository at this point in the history

  23. Add documentation

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    3f2eb58 View commit details
    Browse the repository at this point in the history

  24. get extra spaces back

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    7fe0384 View commit details
    Browse the repository at this point in the history

  25. Improve error message

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    e00a9be View commit details
    Browse the repository at this point in the history

  26. Fix translation files after merge

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    60239b6 View commit details
    Browse the repository at this point in the history

  27. fix translation files 2

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    45279ae View commit details
    Browse the repository at this point in the history

  28. Add zh translation for import functionality

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    491967c View commit details
    Browse the repository at this point in the history

  29. Sync mailisearch index after import

    @DenisPalnitsky @danny-avila
    DenisPalnitsky authored and danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    fe2ed86 View commit details
    Browse the repository at this point in the history

  30. chore: add dummy uri for jest tests, as MONGO_URI should only be real… 

    … for E2E tests
    @danny-avila
    danny-avila committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    7116daf View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. Merge branch 'main' into feature/chats-import

    @DenisPalnitsky
    DenisPalnitsky committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    008c81c View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. Merge remote-tracking branch 'librechat/main' into feature/chats-import

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    fe40a5f View commit details
    Browse the repository at this point in the history

  2. docs: fix links

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    1fdb6cf View commit details
    Browse the repository at this point in the history

  3. docs: fix conversationsImport section

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    61ac76c View commit details
    Browse the repository at this point in the history

  4. fix: user role issue for librechat imports

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    84518da View commit details
    Browse the repository at this point in the history

  5. refactor: import conversations from json 

    - organize imports
- add additional jsdocs
- use multer with diskStorage to avoid loading file into memory outside of job
- use filepath instead of loading data string for imports
- replace console logs and some logger.info() with logger.debug
- only use multer for import route
    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    1ce8e8e View commit details
    Browse the repository at this point in the history

  6. fix: undefined metadata edge case and replace ChatGtp -> ChatGpt

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    5ddfa53 View commit details
    Browse the repository at this point in the history

  7. Refactor importChatGptConvo function to handle undefined metadata edg… 

    …e case and replace ChatGtp with ChatGpt
    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    9ff802b View commit details
    Browse the repository at this point in the history

  8. fix: chatgpt importer

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    73edea9 View commit details
    Browse the repository at this point in the history

  9. feat: maintain tree relationship for librechat messages

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    a66a4d1 View commit details
    Browse the repository at this point in the history

  10. chore: use enum

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    3a2e33d View commit details
    Browse the repository at this point in the history

  11. refactor: saveMessage to use single object arg, replace console logs,… 

    … add userId to log message
    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    63e1e0d View commit details
    Browse the repository at this point in the history

  12. chore: additional comment

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    0fbe3d2 View commit details
    Browse the repository at this point in the history

  13. chore: multer edge case

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    327506c View commit details
    Browse the repository at this point in the history

  14. feat: first pass, maintain tree relationship

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    cd9bd1e View commit details
    Browse the repository at this point in the history

  15. chore: organize

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    7951b55 View commit details
    Browse the repository at this point in the history

  16. chore: remove log

    @danny-avila
    danny-avila committed May 1, 2024
    Configuration menu
    Copy the full SHA
    f9e161d View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. ci: add heirarchy test for chatgpt

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    f406aaa View commit details
    Browse the repository at this point in the history

  2. ci: test maintaining of heirarchy for librechat

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    dfc8895 View commit details
    Browse the repository at this point in the history

  3. wip: allow non-text content type messages

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    00c54e1 View commit details
    Browse the repository at this point in the history

  4. refactor: import content part object json string

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    8be339a View commit details
    Browse the repository at this point in the history

  5. refactor: more content types to format

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    ade6971 View commit details
    Browse the repository at this point in the history

  6. chore: consolidate messageText formatting

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    8e7f458 View commit details
    Browse the repository at this point in the history

  7. docs: update on changes, bump data-provider/config versions, update r… 

    …eadme
    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    38c9db5 View commit details
    Browse the repository at this point in the history

  8. refactor(indexSync): singleton pattern for MeiliSearchClient

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    edef308 View commit details
    Browse the repository at this point in the history

  9. refactor: debug log after batch is done

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    137c7b3 View commit details
    Browse the repository at this point in the history

  10. chore: add back indexSync error handling

    @danny-avila
    danny-avila committed May 2, 2024
    Configuration menu
    Copy the full SHA
    2a486a6 View commit details
    Browse the repository at this point in the history