-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[8.14 QA] Validate connector client #2446
Comments
@khushbu-elastic instead of the steps described above, please test these 2 connectors following the public documentation forConnectors API and Connectors CLI |
We have referred to the documentation for Connectors API & Connectors CLI and below are our findings: Connectors API:
Connectors CLI:
We think that the documentation needs to be updated accordingly for better understanding of the users. |
https://www.elastic.co/guide/en/elasticsearch/reference/current/create-connector-api.html -> index name is optional, we will update the documentation in the next release Is there another place where you were looking into the documentation? |
@khushbu-elastic for the Connectors CLI test case, have you followed the steps here? Which connector have you tried to create? Just to confirm it was not a native connector converted to connector client? |
We are checking for Connector API here only.
We are following the same steps Connector CLI Also while executing Connector API, we have to generate the API Key from UI, which also needs to be added in the steps of Connector API. CC: @vidok |
Hey @khushbu-elastic, in
The framework will take care of creating the index with appropriate mappings upon the first sync. If you are confused about certain steps please provide us with feedback, and feel free to reach out to me for help with the APIs. Also, in a matter of days we should have a tutorial in our docs that should make using a APIs a bit more clear ;) |
@jedrazb Thanks for sharing the updated API documentation for Create Connector API. We had referred to the documentation and tried to Create Connector via API in and with
so we are not sure how will it work with request body being Optional (as mentioned in the documentation). It would be great if you can share a demo for Connector API steps for better understanding. |
Thank you for your feedback! I addressed it in this PR. |
Hey @khushbu-elastic , those errors are an obfuscated way that your request body is empty. We should add better error messages when the request body for connector creation is empty - I will create a ticket to handle this. You should pass the request body to the create connector endpoints as in documentation, have you tried examples from the documentation? For now you can refer to this tutorial preview that will be released soon https://www.elastic.co/guide/en/enterprise-search/master/connectors-tutorial-api.html |
Also, you can refer to the postman collection with all API endpoints here: https://github.com/elastic/connectors/tree/main/resources/connectors_api |
We tried executing the Connector API from the updated documentation but still facing the same issue of Index not getting created/attached and we have create/attach it from the UI in order to map it with Connector via Connector API. Index mapping is still mandatory (in Elastic v-8.14-latest). PFA @jedrazb Could you please share the demo for Connector API with steps for better understanding. |
Hey @khushbu-elastic we have update index name endpoint that attached the index name to the connector: https://www.elastic.co/guide/en/elasticsearch/reference/master/update-connector-index-name-api.html All the available endpoints are listed in: https://www.elastic.co/guide/en/elasticsearch/reference/master/connector-apis.html |
@jedrazb Okay, we can refer that when we will check for Index creation and attachment. But if we are treating the index-name as an optional value, we should be able to move further without creating/attaching index to connector, according to my understanding. Please let me know if we are not on same page for considering the index-name as an optional value. |
@khushbu-elastic there might be a gap in Kibana that causes this issue when index doesn't exist yet. You should be able to get around it when:
|
We have verified the Postgresql & ABS as Connectors API and Connectors CLI. Below points are our observations:
|
@DianaJourdan Please let us know if there is any update on this, considering the above comment. |
Non-regression QA
Note: always test with the latest Build Candidate on Elastic Cloud, using the full Elastic stack
Connector name and description are editable on the Configurations page
Connector can be deleted from the Indices page
Connector can be deleted from the Indices page and it can be recreated with the same name after
Pull connectors repository, run
make install
but do not run connector yetVerify that you are redirected to "configuration" page where you can create an api key and can copy connector id / whole section of config into the connector
Update connector configuration with the api_key and connector_id, plus choose a service_type to test and set it in config
Start the connector by running
make run
- verify that it starts and does not actually do anything yetWait for the Kibana page with the connector configuration to update and verify that it's possible to edit connector configuration now
Edit and save connector configuration, then reload the page and verify that configuration is properly saved
Click on "Set schedule and sync" and verify that you're redirected to the scheduling tab
Enable scheduling for frequency = every minute and save schedule; refresh the page and verify that the changes were stored
Switch to the connector and wait for a minute or two, verify that connector starts to ingest data
Verify that the data from the connector appears in the expected index
Verify that on the index list page index information is updated properly, showing expected number of documents and new index size
Verify that on the connector overview page "Document Count" is updated to reflect the number of documents in the index
Verify that you can see ingested documents in
documents
tabVerify that index mappings are correct on the
index mappings
tabRecord a short demo showing the connectors' configuration and that there were documents ingested
The text was updated successfully, but these errors were encountered: