Enable support for custom filesystem

[thiagosalvatore]

Custom FileSystems that are now available on llama-index were not available on llama-parse. We can now use it to download files using any reader like S3Reader.

Fixes #116

[thiagosalvatore]

@logan-markewich can you take a look at this when you have some time, please?

[logan-markewich]

I don't think this works, at least locally, it doesn't work for me

Since we already have the path, can't we use that?

[thiagosalvatore]

I am using it in production right now and it is working properly. What is the error that you are getting locally?

The issue with the path is that I've seen situations in production where people upload a file with .txt but the file is actually a .csv and it doesn't work.

[sheresaidon]

@logan-markewich whenever you have a chance could you please review, currently llama parse is giving me the fs error when trying to use it with GCS Reader

parser = LlamaParse(result_type="markdown", api_key=LLAMA_PARSE_API_KEY,verbose=True, show_progress=True)

reader = GCSReader(
    bucket=bucket_name,
    key=file_path,  # Use the extracted file path
    file_extractor={".pdf": parser, ".docx": parser, ".txt": parser},
    service_account_key=service_account_key_dict
)

LlamaParse.load_data() got an unexpected keyword argument 'fs'. Skipping...