Limiting response

[jeremyzarge]

Is there a way to limit Bard's response to a few sentences? Can this be done with some sort of max_tokens parameter?

[dsdanielpark]

Bard does not support temperature or hyperparameter adjustments, but it is possible to achieve the appearance of limiting the number of output tokens or the number of output sentences using simple algorithms, as follows:

pip install -U git+https://github.com/dsdanielpark/Bard-API.git

from bardapi import Bard, max_token, max_sentence

token = 'xxxxxxx'
bard = Bard(token=token)
max_token(bard.get_answer("나와 내 동년배들이 좋아하는 뉴진스에 대해서 알려줘")['content'], 30)

def max_token(text: str, n: int):
    """
    Print the first 'n' tokens (words) of the given text.

    Args:
        text (str): The input text to be processed.
        n (int): The number of tokens (words) to be printed from the beginning.

    Returns:
        None
    """
    word_count = 0
    word_start = 0
    for i, char in enumerate(text):
        if char.isspace():
            word_count += 1
            if word_count == n:
                print(text[:i])
                break
    else:
        print(text)


def max_sentence(text: str, n: int):
    """
    Print the first 'n' sentences of the given text.

    Args:
        text (str): The input text to be processed.
        n (int): The number of sentences to be printed from the beginning.

    Returns:
        None
    """
    punctuations = set('?!.')

    sentences = []
    sentence_count = 0
    for char in text:
        sentences.append(char)
        if char in punctuations:
            sentence_count += 1
            if sentence_count == n:
                print(''.join(sentences).strip())
                return
    print(''.join(sentences).strip())


text = "Is there a way to limit Bard's response to a few sentences? Can this be done with some sort of max_tokens parameter? I want to ask something."
max_sentence(text, 2)

[jeremyzarge]

Correct, but in this way it doesn't actually change the way Bard is responding, it merely just cuts if off after a certain amount of sentences, meaning it's not necessarily a shortened version of the answer.

[dsdanielpark]

That's correct. However, currently, bardapi cannot appropriately modify the config for generative AI. Please refer to Google's PALM API (if you are on their whitelist). But still, it may not be feasible to provide resources to set up the config for generative AI for regular users.

[cumaron]

Hello, you can limit the response by specifying the number of words in your query, for example 'Explain in 200 words ...".