Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PLUGIN-1837] Error management for Analytics plugin i.e. GroupByAggregate, Deduplicate, Distinct and Joiner #1903

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

Amit-CloudSufi
Copy link

@Amit-CloudSufi Amit-CloudSufi commented Jan 8, 2025

Jira : PLUGIN-1837

Test [ Distinct Aggregator ]

  • Test Case (Enter a field that does not exist in input schema)
Raw Logs
Caused by: io.cdap.cdap.api.exception.WrappedStageException: Stage 'Distinct' encountered : io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Field age_fake does not exist in input schema.
  at io.cdap.cdap.etl.common.plugin.ExceptionWrappingCaller.call(ExceptionWrappingCaller.java:64)
  at io.cdap.cdap.etl.common.plugin.WrappedReduceAggregator.prepareRun(WrappedReduceAggregator.java:80)
  at io.cdap.cdap.etl.common.plugin.WrappedReduceAggregator.prepareRun(WrappedReduceAggregator.java:36)
  at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:74)
  at io.cdap.cdap.internal.app.runtime.AbstractContext.lambda$execute$5(AbstractContext.java:558)
  at io.cdap.cdap.data2.transaction.Transactions$CacheBasedTransactional.finishExecute(Transactions.java:234)
  ... 21 common frames omitted
Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Field age_fake does not exist in input schema.
  at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:78)
  at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50)
  at io.cdap.plugin.batch.aggregator.DistinctAggregator.prepareRun(DistinctAggregator.java:114)
  at io.cdap.cdap.etl.common.plugin.WrappedReduceAggregator.lambda$prepareRun$3(WrappedReduceAggregator.java:81)
  at io.cdap.cdap.etl.common.plugin.Caller$1.call(Caller.java:30)
  at io.cdap.cdap.etl.common.plugin.ExceptionWrappingCaller.call(ExceptionWrappingCaller.java:62)
  ... 26 common frames omitted

POST v3/namespaces/{namespace-id}/apps/{app-id}/workflows/DataPipelineWorkflow/runs/{run-id}/classify
[
{
  "stageName": "Distinct",
  "errorCategory": "Plugin-'Validation'-'Distinct'",
  "errorReason": "Stage 'Distinct' encountered 1 validation failures.",
  "errorMessage": "Stage 'Distinct' encountered : io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Field age_fake does not exist in input schema.",
  "errorType": "USER",
  "dependency": "false"
}
]

image
image

@psainics psainics added the build Trigger unit test build label Jan 8, 2025
@Amit-CloudSufi Amit-CloudSufi force-pushed the hydratorPluginErrorMang branch from c0f275e to 3b3b46f Compare January 8, 2025 12:28
@psainics psainics changed the title Error management for Analytics plugin i.e. GroupByAggregate, Deduplicate, Distinct and Joiner [PLUGIN-1837] Error management for Analytics plugin i.e. GroupByAggregate, Deduplicate, Distinct and Joiner Jan 8, 2025
@Amit-CloudSufi Amit-CloudSufi force-pushed the hydratorPluginErrorMang branch from 27ba401 to 9c5a314 Compare January 9, 2025 07:49
@Amit-CloudSufi Amit-CloudSufi force-pushed the hydratorPluginErrorMang branch from 2cb74cb to 60c6dee Compare January 9, 2025 09:53
@psainics psainics marked this pull request as ready for review January 9, 2025 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Trigger unit test build
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants