Validator.java rejecting certain custom datatypes

[rosita-hormann]

While using KairosDB in my team, we decided to add a custom datatype to make our kairosdb database accept ArrayTypes. The plugin was implemented and added to KairosDB, but we encountered a problem while trying to insert custom data to the database.

We sent this json to KairosDB API to isnert via Python:

{
    "name": "example-array",
    "tags": {
        "test": "true"
        },
    "type": "arraynumber",
    "datapoints": [
            [1554076801000, [458, 1, 1, 1]]
        ]
}

The API returned the following message:

result.status_code 500
result.text {"errors":[null]}
result.reason Internal Server Error

When checking KairosDB logs, we encountered these error messages:

18:35:58.773 [qtp3392189-28] ERROR [MetricsResource.java:396] - Failed to add metric.
java.lang.IllegalStateException: null
    at com.google.gson.JsonArray.getAsString(JsonArray.java:133)
    at org.kairosdb.util.Validator.isNotNullOrEmpty(Validator.java:87)
    at org.kairosdb.core.http.rest.json.DataPointsParser.validateAndAddDataPoints(DataPointsParser.java:361)
    at org.kairosdb.core.http.rest.json.DataPointsParser.parse(DataPointsParser.java:108)
    at org.kairosdb.core.http.rest.MetricsResource.add(MetricsResource.java:372)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
    at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
    at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
    at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
    at org.kairosdb.core.http.MonitorFilter.doFilter(MonitorFilter.java:81)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at org.kairosdb.core.http.LoggingFilter.doFilter(LoggingFilter.java:48)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
    at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
    at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:748)

After debugging KairosDB, we found that the exception was being thrown on Validator.json in this funcion, specifically where I put the comment "HERE":

	public static boolean isNotNullOrEmpty(ValidationErrors validationErrors, Object name, JsonElement value)
	{
		if (value == null)
		{
			validationErrors.addErrorMessage(name + " may not be null.");
			return false;
		}
		if (value.isJsonNull())
		{
			validationErrors.addErrorMessage(name + " may not be empty.");
			return false;
		}
		if (value.isJsonArray() && value.getAsJsonArray().size() < 1)
		{ // HERE
			validationErrors.addErrorMessage(name + " may not be an empty array.");
			return false;
		}
		if (!value.isJsonObject() && value.getAsString().isEmpty())
		{ // and HERE
			validationErrors.addErrorMessage(name + " may not be empty.");
			return false;
		}

		return true;
	}

In summary, KairosDB does not recognize our custom data type "ArrayNumbers" as JsonArray, instead it considers it as JsonObject and, because it's empty, it returns false and then it throws java.lang.IllegalStateException. The behaviour we would expect is that the custom ArrayType falls under the category JsonArray and if it's not emty, return as a valid type right away.

After doing this change, the problem was solved:

    // ...
		if (value.isJsonArray()) {
			if (value.getAsJsonArray().size() < 1) {
				validationErrors.addErrorMessage(name + " may not be an empty array.");
				return false;
			}
			return true;
		}

And here is an example of what KairosDB returns after inserting our custom datatype after the fix:

[
    {
        "sample_size":1,
        "results":[
            {
                "name":"example-array",
                "group_by":[{"name":"type","type":"array"}],
                "tags":{"test":["true"]},
                "values":[
                        [1554076800000,[128,1,1,1]]
                    ]
            }
        ]
    }
]

We propose this to be added to the current stable version of KairosDB (v1.2.2)

Reference: https://github.com/javarias/kairosdb/tree/v1.2.2-patched