09-01-2023 15:11:36.701 INFO dispatchRunner [1791713 MainThread] - Search process mode: preforked (reused process) (build 64e843ea36b1). 09-01-2023 15:11:36.702 INFO dispatchRunner [1791713 MainThread] - registering build time modules, count=1 09-01-2023 15:11:36.702 INFO dispatchRunner [1791713 MainThread] - registering search time components of build time module name=vix 09-01-2023 15:11:36.702 INFO BundlesSetup [1791713 MainThread] - Setup stats for /opt/splunk/etc: wallclock_elapsed_msec=13, cpu_time_used=0.0132636, shared_services_generation=2, shared_services_population=1 09-01-2023 15:11:36.707 INFO UserManagerPro [1791713 MainThread] - Load authentication: forcing roles="admin, power, user" 09-01-2023 15:11:36.708 INFO UserManager [1792851 RunDispatch] - Setting user context: admin 09-01-2023 15:11:36.708 INFO UserManager [1792851 RunDispatch] - Done setting user context: NULL -> admin 09-01-2023 15:11:36.708 INFO dispatchRunner [1792851 RunDispatch] - search context: user="admin", app="search", bs-pathname="/opt/splunk/etc" 09-01-2023 15:11:36.708 INFO SearchParser [1792851 RunDispatch] - PARSING: search index=hadoop 09-01-2023 15:11:36.708 INFO dispatchRunner [1792851 RunDispatch] - Search running in non-clustered mode 09-01-2023 15:11:36.708 INFO dispatchRunner [1792851 RunDispatch] - SearchHeadInitSearchMs=0 09-01-2023 15:11:36.708 INFO dispatchRunner [1792851 RunDispatch] - Executing the Search orchestrator and iterator model (dfs=false). 09-01-2023 15:11:36.709 INFO SearchOrchestrator [1792851 RunDispatch] - SearchOrchestrator is constructed. sid=1693573896.13, eval_only=0 09-01-2023 15:11:36.709 INFO SearchOrchestrator [1792851 RunDispatch] - Initialized the SRI 09-01-2023 15:11:36.709 INFO SearchFeatureFlags [1792851 RunDispatch] - Initializing feature flags from config. feature_seed=3903644489 09-01-2023 15:11:36.709 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=parallelreduce:enablePreview:true 09-01-2023 15:11:36.709 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=search:search_retry:false 09-01-2023 15:11:36.709 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=search:search_retry_realtime:false 09-01-2023 15:11:36.709 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=parallelreduce:autoAppliedPercentage:false 09-01-2023 15:11:36.709 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=subsearch:enableConcurrentPipelineProcessing:false 09-01-2023 15:11:36.709 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=subsearch:concurrent_pipeline_adhoc:false 09-01-2023 15:11:36.710 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=append:support_multiple_data_sources:false 09-01-2023 15:11:36.710 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=join:support_multiple_data_sources:false 09-01-2023 15:11:36.710 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=stats:allow_stats_v2:true 09-01-2023 15:11:36.710 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=search_optimization::set_required_fields:stats:false 09-01-2023 15:11:36.710 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=searchresults:srs2:false 09-01-2023 15:11:36.710 INFO SearchFeatureFlags [1792851 RunDispatch] - Setting feature_flag=search:read_final_results_from_timeliner:false 09-01-2023 15:11:36.710 INFO SearchOrchestrator [1792851 RunDispatch] - Search feature_flags={"v":1,"enabledFeatures":["parallelreduce:enablePreview","stats:allow_stats_v2"],"disabledFeatures":["search:search_retry","search:search_retry_realtime","parallelreduce:autoAppliedPercentage","subsearch:enableConcurrentPipelineProcessing","subsearch:concurrent_pipeline_adhoc","append:support_multiple_data_sources","join:support_multiple_data_sources","search_optimization::set_required_fields:stats","searchresults:srs2","search:read_final_results_from_timeliner"]} 09-01-2023 15:11:36.710 INFO ISplunkDispatch [1792851 RunDispatch] - Not running in splunkd. Bundle replication not triggered. 09-01-2023 15:11:36.710 INFO SearchOrchestrator [1792854 searchOrchestrator] - Initialzing the run time settings for the orchestrator. 09-01-2023 15:11:36.710 INFO UserManager [1792854 searchOrchestrator] - Setting user context: admin 09-01-2023 15:11:36.710 INFO UserManager [1792854 searchOrchestrator] - Done setting user context: NULL -> admin 09-01-2023 15:11:36.710 INFO AdaptiveSearchEngineSelector [1792854 searchOrchestrator] - Search execution_plan=classic 09-01-2023 15:11:36.710 INFO SearchOrchestrator [1792854 searchOrchestrator] - Creating the search DAG. 09-01-2023 15:11:36.710 INFO SearchParser [1792854 searchOrchestrator] - PARSING: search index=hadoop 09-01-2023 15:11:36.710 INFO DispatchStorageManagerInfo [1792854 searchOrchestrator] - Successfully created new dispatch directory for search job. sid=da3907068217d120_tmp dispatch_dir=/opt/splunk/var/run/splunk/dispatch/da3907068217d120_tmp 09-01-2023 15:11:36.720 INFO SearchProcessor [1792854 searchOrchestrator] - Building search filter 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing lookup expansions 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Lookup expansion took 0 ms 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing calculated field expansions 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Calculated field expansion took 0 ms 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing field alias expansions 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - field alias expansions took 0 ms 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing kvfield expansions 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - kvfield expansions took 0 ms 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing indexed fields expansions 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - indexed fields expansions took 0 ms 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing reverse sourcetype expansions 09-01-2023 15:11:36.750 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - sourcetype expansions took 0 ms 09-01-2023 15:11:36.750 INFO UnifiedSearch [1792854 searchOrchestrator] - Expanded index search = index=hadoop 09-01-2023 15:11:36.750 INFO UnifiedSearch [1792854 searchOrchestrator] - base lispy: [ AND index::hadoop ] 09-01-2023 15:11:36.751 INFO UnifiedSearch [1792854 searchOrchestrator] - Processed search targeting arguments 09-01-2023 15:11:36.751 INFO DispatchThread [1792854 searchOrchestrator] - Disabling remote timeline computation due to processor name=search, not allowing it 09-01-2023 15:11:36.751 INFO DispatchThread [1792854 searchOrchestrator] - BatchMode: allowBatchMode: 0, conf(1): 1, timeline/Status buckets(0):0, realtime(0):0, report pipe empty(0):1, reqTimeOrder(0):0, summarize(0):0, statefulStreaming(0):0 09-01-2023 15:11:36.751 INFO DispatchThread [1792854 searchOrchestrator] - Setup timeliner partialCommits=0 09-01-2023 15:11:36.751 INFO DispatchThread [1792854 searchOrchestrator] - required fields list to add to remote search = _bkt,_cd,_si,host,index,linecount,source,sourcetype,splunk_server 09-01-2023 15:11:36.751 INFO SearchParser [1792854 searchOrchestrator] - PARSING: fields keepcolorder=t "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:36.751 INFO DispatchCommandProcessor [1792854 searchOrchestrator] - summaryHash=212034b4d5359d29 summaryId=BAB88FE8-7AA9-4F61-AF10-41B5EF52949C_search_admin_212034b4d5359d29 remoteSearch=litsearch index=hadoop | fields keepcolorder=t "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:36.751 INFO DispatchCommandProcessor [1792854 searchOrchestrator] - summaryHash=NS878cdcdfffddf849 summaryId=BAB88FE8-7AA9-4F61-AF10-41B5EF52949C_search_admin_NS878cdcdfffddf849 remoteSearch=litsearch index=hadoop | fields keepcolorder=t "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:36.751 INFO DispatchThread [1792854 searchOrchestrator] - Getting summary ID for summaryHash=NS878cdcdfffddf849 09-01-2023 15:11:36.752 INFO SearchParser [1792854 searchOrchestrator] - PARSING: search index=hadoop 09-01-2023 15:11:36.752 INFO UnifiedSearch [1792854 searchOrchestrator] - Processed search targeting arguments 09-01-2023 15:11:36.752 INFO AstOptimizer [1792854 searchOrchestrator] - SrchOptMetrics optimize_toJson=0.000531857 09-01-2023 15:11:36.752 INFO AstVisitorFactory [1792854 searchOrchestrator] - Not building visitor : replace_datamodel_stats_cmds_with_tstats 09-01-2023 15:11:36.752 INFO SearchParser [1792854 searchOrchestrator] - PARSING: | search index=hadoop 09-01-2023 15:11:36.752 INFO FederatedInfo [1792854 searchOrchestrator] - No federated search providers defined. 09-01-2023 15:11:36.752 INFO ReplaceTableWithFieldsVisitor [1792854 searchOrchestrator] - search_optimization::replace_table_with_fields disabled due to VERBOSE Mode search 09-01-2023 15:11:36.752 INFO AstVisitorFactory [1792854 searchOrchestrator] - Not building visitor : replace_table_with_fields 09-01-2023 15:11:36.752 INFO ProjElim [1792854 searchOrchestrator] - Black listed processors=[addinfo] 09-01-2023 15:11:36.753 INFO AstVisitorFactory [1792854 searchOrchestrator] - Not building visitor : replace_stats_cmds_with_tstats 09-01-2023 15:11:36.753 INFO AstVisitorFactory [1792854 searchOrchestrator] - Not building visitor : replace_chart_cmds_with_tstats 09-01-2023 15:11:36.753 INFO AstOptimizer [1792854 searchOrchestrator] - SrchOptMetrics optimization=0.000392400 09-01-2023 15:11:36.753 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Optimized Search =| search index=hadoop 09-01-2023 15:11:36.753 INFO ScopedTimer [1792854 searchOrchestrator] - search.optimize 0.001459750 09-01-2023 15:11:36.753 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Data federation is enabled 09-01-2023 15:11:36.753 INFO FederatedProviderVisitor [1792854 searchOrchestrator] - Federated Whole Search Remote Execution Enabled: false 09-01-2023 15:11:36.754 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Number of involved federated deployments is 0. 09-01-2023 15:11:36.754 INFO ParallelReducePolicy [1792854 searchOrchestrator] - Current Search Head doesn't have any usable peers to use. 09-01-2023 15:11:36.754 INFO PhaseToPipelineVisitor [1792854 searchOrchestrator] - Phase Search = | search index=hadoop 09-01-2023 15:11:36.754 INFO SearchParser [1792854 searchOrchestrator] - PARSING: | search index=hadoop 09-01-2023 15:11:36.763 INFO SearchProcessor [1792854 searchOrchestrator] - Building search filter 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing lookup expansions 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Lookup expansion took 0 ms 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing calculated field expansions 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Calculated field expansion took 0 ms 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing field alias expansions 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - field alias expansions took 0 ms 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing kvfield expansions 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - kvfield expansions took 0 ms 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing indexed fields expansions 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - indexed fields expansions took 0 ms 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing reverse sourcetype expansions 09-01-2023 15:11:36.773 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - sourcetype expansions took 0 ms 09-01-2023 15:11:36.773 INFO UnifiedSearch [1792854 searchOrchestrator] - Expanded index search = index=hadoop 09-01-2023 15:11:36.773 INFO UnifiedSearch [1792854 searchOrchestrator] - base lispy: [ AND index::hadoop ] 09-01-2023 15:11:36.774 INFO UnifiedSearch [1792854 searchOrchestrator] - Processed search targeting arguments 09-01-2023 15:11:36.774 INFO PhaseToPipelineVisitor [1792854 searchOrchestrator] - Phase Search = 09-01-2023 15:11:36.774 INFO SearchPipeline [1792854 searchOrchestrator] - ReportSearch=0 AllowBatchMode=0 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Storing only 1000 events per timeline buckets due to limits.conf max_events_per_bucket setting. 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - No need for RTWindowProcessor 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Adding timeliner to final phase 09-01-2023 15:11:36.774 INFO SearchParser [1792854 searchOrchestrator] - PARSING: | timeliner remote=0 partial_commits=0 max_events_per_bucket=1000 fieldstats_update_maxperiod=60 bucket=300 extra_field=* 09-01-2023 15:11:36.774 INFO TimelineCreator [1792854 searchOrchestrator] - Creating timeline with remote=0 partialCommits=0 commitFreq=0 syncKSFreq=0 maxSyncKSPeriodTime=60000 bucket=300 latestTime=1693573896.000000 earliestTime=1693486800.000000 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - required fields list to add to different pipelines = *,_bkt,_cd,_si,host,index,linecount,source,sourcetype,splunk_server 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Remote Timeliner= 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Fileds=fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - REMOTE TIMELINER ADDED 09-01-2023 15:11:36.774 INFO SearchParser [1792854 searchOrchestrator] - PARSING: fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Adding noop to provide result collation at search head 09-01-2023 15:11:36.774 INFO SearchParser [1792854 searchOrchestrator] - PARSING: noop 09-01-2023 15:11:36.774 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Search Phases created. 09-01-2023 15:11:36.775 INFO SearchOrchestrator [1792854 searchOrchestrator] - Preview is enabled for Search Head only. (preview_algorithm=SEARCH_HEAD) 09-01-2023 15:11:36.775 INFO UserManager [1792854 searchOrchestrator] - Setting user context: admin 09-01-2023 15:11:36.775 INFO UserManager [1792854 searchOrchestrator] - Done setting user context: admin -> admin 09-01-2023 15:11:36.775 INFO UserManager [1792854 searchOrchestrator] - Unwound user context: admin -> admin 09-01-2023 15:11:36.775 INFO DistributedSearchResultCollectionManager [1792854 searchOrchestrator] - Stream search: litsearch index=hadoop | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:36.776 INFO ExternalResultProvider [1792854 searchOrchestrator] - Search after asserting splunk_server=local: index=hadoop 09-01-2023 15:11:36.780 WARN ExternalResultProvider [1792854 searchOrchestrator] - SearchMessage orig_component=ExternalResultProvider sid=1693573896.13 message_key= message=Could not get the latest bundle. Reason: Bundle path could not be retrieved 09-01-2023 15:11:36.780 INFO ExternalResultProvider [1792854 searchOrchestrator] - Creating external result provider=MyHadoopProvider, with index.count=1, search="search (index=hadoop) | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"" 09-01-2023 15:11:38.572 INFO SearchParser [1792854 searchOrchestrator] - PARSING: stdin | search (index=hadoop) | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:38.598 INFO SearchParser [1792854 searchOrchestrator] - PARSING: typer | tags 09-01-2023 15:11:38.613 INFO FastTyper [1792854 searchOrchestrator] - found nodes count: comparisons=6, unique_comparisons=5, terms=4, unique_terms=4, phrases=12, unique_phrases=12, total leaves=22 09-01-2023 15:11:38.615 INFO SearchOperator:stdin [1792854 searchOrchestrator] - setting _need_timestamp_fields=0, required time field name= 09-01-2023 15:11:38.615 INFO SearchOperator:stdin [1792854 searchOrchestrator] - required fields list = _raw,_time,host,source,sourcetype 09-01-2023 15:11:38.616 INFO UnifiedSearch [1792854 searchOrchestrator] - Processed search targeting arguments 09-01-2023 15:11:38.628 INFO SearchOperator:stdin [1792854 searchOrchestrator] - setting _need_timestamp_fields=1, required time field name=date_second 09-01-2023 15:11:38.628 INFO SearchOperator:stdin [1792854 searchOrchestrator] - required fields list = *,Message,_bkt,_cd,_raw,_si,_time,host,index,linecount,source,sourcetype,splunk_server 09-01-2023 15:11:38.629 INFO ExternalResultProvider [1792854 searchOrchestrator] - provider=MyHadoopProvider, mode.config=report, mode.search=stream 09-01-2023 15:11:38.629 INFO SearchParser [1792854 searchOrchestrator] - PARSING: search (index=hadoop) | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:38.649 INFO SearchProcessor [1792854 searchOrchestrator] - Building search filter 09-01-2023 15:11:38.661 INFO CsvDataProvider [1792854 searchOrchestrator] - Reading schema for lookup table='dmc_assets', file size=495, modtime=1693572337 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing lookup expansions 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Lookup expansion took 0 ms 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing calculated field expansions 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Calculated field expansion took 0 ms 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing field alias expansions 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - field alias expansions took 0 ms 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing kvfield expansions 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - kvfield expansions took 0 ms 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing indexed fields expansions 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - indexed fields expansions took 0 ms 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - Performing reverse sourcetype expansions 09-01-2023 15:11:38.686 INFO SearchEvaluatorBasedExpander [1792854 searchOrchestrator] - sourcetype expansions took 0 ms 09-01-2023 15:11:38.686 INFO UnifiedSearch [1792854 searchOrchestrator] - Expanded index search = index=hadoop 09-01-2023 15:11:38.686 INFO UnifiedSearch [1792854 searchOrchestrator] - base lispy: [ AND index::hadoop ] 09-01-2023 15:11:38.688 INFO UnifiedSearch [1792854 searchOrchestrator] - Initialization of search data structures took 1 ms 09-01-2023 15:11:38.689 INFO UnifiedSearch [1792854 searchOrchestrator] - Processed search targeting arguments 09-01-2023 15:11:38.689 INFO ERP.MyHadoopProvider [1792854 searchOrchestrator] - Starting: /opt/splunk/bin/jars/sudobash /opt/hadoop/bin/hadoop jar "/opt/splunk/bin/jars/SplunkMR-hy3.jar" "com.splunk.mr.SplunkMR" 09-01-2023 15:11:38.690 INFO DistributedSearchResultCollectionManager [1792854 searchOrchestrator] - ERP providers detected, hence collation will follow unordered/round robin mode. 09-01-2023 15:11:38.690 INFO DistributedSearchResultCollectionManager [1792854 searchOrchestrator] - ERP providers detected, safeguards against searches waiting indefinately for new results have been removed 09-01-2023 15:11:38.690 INFO DistributedSearchResultCollectionManager [1792854 searchOrchestrator] - results_queue_read_timeout_sec=2147483647.000 send_timeout=30.000 connect_timeout=10.000 receive_timeout=600.000 09-01-2023 15:11:38.690 INFO DistributedSearchResultCollectionManager [1792854 searchOrchestrator] - Default search group:* 09-01-2023 15:11:38.690 INFO SearchTargeter [1792854 searchOrchestrator] - Index list is not available for tst-hadoop-mn01 targeting based on splunk_server and splunk_server_group only 09-01-2023 15:11:38.690 INFO DistributedSearchResultCollectionManager [1792854 searchOrchestrator] - Connecting to peer tst-hadoop-mn01 connectAll 0 connectToSpecificPeer 1 09-01-2023 15:11:38.690 INFO UserManager [1792960 SearchResultExecutorThread] - Setting user context: splunk-system-user 09-01-2023 15:11:38.690 INFO UserManager [1792960 SearchResultExecutorThread] - Done setting user context: NULL -> splunk-system-user 09-01-2023 15:11:38.690 INFO SearchPhaseGenerator [1792854 searchOrchestrator] - Time spends on creating distributed search results infrastructure; dispatchcreatedSearchResultInfrastructure=1.914932405 seconds. 09-01-2023 15:11:38.690 INFO UserManager [1792960 SearchResultExecutorThread] - Unwound user context: splunk-system-user -> NULL 09-01-2023 15:11:38.690 INFO UserManager [1792960 SearchResultExecutorThread] - Setting user context: admin 09-01-2023 15:11:38.690 INFO UserManager [1792960 SearchResultExecutorThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.690 INFO UserManager [1792958 erpCollector] - Setting user context: splunk-system-user 09-01-2023 15:11:38.690 INFO UserManager [1792958 erpCollector] - Done setting user context: NULL -> splunk-system-user 09-01-2023 15:11:38.690 INFO SearchOrchestrator [1792854 searchOrchestrator] - Starting the status control thread. 09-01-2023 15:11:38.690 INFO SearchOrchestrator [1792854 searchOrchestrator] - Starting phase=1 09-01-2023 15:11:38.690 INFO UserManager [1792965 localCollectorThread] - Setting user context: admin 09-01-2023 15:11:38.690 INFO UserManager [1792964 phase_1] - Setting user context: admin 09-01-2023 15:11:38.690 INFO UserManager [1792964 phase_1] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.690 INFO ReducePhaseExecutor [1792964 phase_1] - Starting phase_1 09-01-2023 15:11:38.690 INFO UserManager [1792856 StatusEnforcerThread] - Setting user context: admin 09-01-2023 15:11:38.690 INFO UserManager [1792856 StatusEnforcerThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.690 INFO UserManager [1792958 erpCollector] - Unwound user context: splunk-system-user -> NULL 09-01-2023 15:11:38.690 INFO UserManager [1792965 localCollectorThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.690 INFO SearchStatusEnforcer [1792856 StatusEnforcerThread] - Enforcing disk quota = 10485760000 09-01-2023 15:11:38.690 INFO UserManager [1792958 erpCollector] - Setting user context: admin 09-01-2023 15:11:38.690 INFO UserManager [1792958 erpCollector] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.690 INFO SearchParser [1792965 localCollectorThread] - PARSING: litsearch index=hadoop | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" 09-01-2023 15:11:38.690 INFO ERPSearchResultCollector [1792958 erpCollector] - ERP peer=MyHadoopProvider starts reading search results. 09-01-2023 15:11:38.690 INFO UserManager [1792963 SearchResultExecutorThread] - Setting user context: splunk-system-user 09-01-2023 15:11:38.691 INFO UserManager [1792963 SearchResultExecutorThread] - Done setting user context: NULL -> splunk-system-user 09-01-2023 15:11:38.691 INFO PreviewExecutor [1792856 StatusEnforcerThread] - Preview Enforcing initialization done 09-01-2023 15:11:38.691 INFO ReducePhaseExecutor [1792856 StatusEnforcerThread] - ReducePhaseExecutor=1 action=PREVIEW 09-01-2023 15:11:38.691 INFO UserManager [1792963 SearchResultExecutorThread] - Unwound user context: splunk-system-user -> NULL 09-01-2023 15:11:38.691 INFO UserManager [1792963 SearchResultExecutorThread] - Setting user context: admin 09-01-2023 15:11:38.691 INFO UserManager [1792963 SearchResultExecutorThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.691 INFO UserManager [1792961 SearchResultExecutorThread] - Setting user context: splunk-system-user 09-01-2023 15:11:38.691 INFO UserManager [1792961 SearchResultExecutorThread] - Done setting user context: NULL -> splunk-system-user 09-01-2023 15:11:38.691 INFO UserManager [1792961 SearchResultExecutorThread] - Unwound user context: splunk-system-user -> NULL 09-01-2023 15:11:38.691 INFO UserManager [1792961 SearchResultExecutorThread] - Setting user context: admin 09-01-2023 15:11:38.691 INFO UserManager [1792961 SearchResultExecutorThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.691 INFO UserManager [1792959 SearchResultExecutorThread] - Setting user context: splunk-system-user 09-01-2023 15:11:38.691 INFO UserManager [1792959 SearchResultExecutorThread] - Done setting user context: NULL -> splunk-system-user 09-01-2023 15:11:38.691 INFO UserManager [1792959 SearchResultExecutorThread] - Unwound user context: splunk-system-user -> NULL 09-01-2023 15:11:38.691 INFO UserManager [1792959 SearchResultExecutorThread] - Setting user context: admin 09-01-2023 15:11:38.691 INFO UserManager [1792959 SearchResultExecutorThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.691 INFO UserManager [1792962 SearchResultExecutorThread] - Setting user context: splunk-system-user 09-01-2023 15:11:38.691 INFO UserManager [1792962 SearchResultExecutorThread] - Done setting user context: NULL -> splunk-system-user 09-01-2023 15:11:38.691 INFO UserManager [1792962 SearchResultExecutorThread] - Unwound user context: splunk-system-user -> NULL 09-01-2023 15:11:38.691 INFO UserManager [1792962 SearchResultExecutorThread] - Setting user context: admin 09-01-2023 15:11:38.691 INFO UserManager [1792962 SearchResultExecutorThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.696 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - MAPREDUCE_USER=, SPLUNK_HOME=/opt/splunk, HADOOP_CLASSPATH= 09-01-2023 15:11:38.704 INFO SearchParser [1792965 localCollectorThread] - PARSING: typer | tags 09-01-2023 15:11:38.706 INFO FastTyper [1792965 localCollectorThread] - found nodes count: comparisons=6, unique_comparisons=5, terms=4, unique_terms=4, phrases=12, unique_phrases=12, total leaves=22 09-01-2023 15:11:38.707 INFO UnifiedSearch [1792965 localCollectorThread] - Initialization of search data structures took 3 ms 09-01-2023 15:11:38.707 INFO UnifiedSearch [1792965 localCollectorThread] - Processed search targeting arguments 09-01-2023 15:11:38.707 INFO LocalCollector [1792965 localCollectorThread] - Final required fields list = *,_bkt,_cd,_raw,_si,_time,host,index,linecount,source,sourcetype,splunk_server 09-01-2023 15:11:38.707 INFO UserManager [1792965 localCollectorThread] - Unwound user context: admin -> NULL 09-01-2023 15:11:38.707 INFO UserManager [1792965 localCollectorThread] - Setting user context: admin 09-01-2023 15:11:38.707 INFO UserManager [1792965 localCollectorThread] - Done setting user context: NULL -> admin 09-01-2023 15:11:38.707 INFO UserManager [1792965 localCollectorThread] - Unwound user context: admin -> NULL 09-01-2023 15:11:38.708 INFO SearchParser [1792856 StatusEnforcerThread] - PARSING: | streamnoop 09-01-2023 15:11:38.708 INFO SearchParser [1792856 StatusEnforcerThread] - PARSING: streamnoop | timeliner remote=0 partial_commits=0 max_events_per_bucket=1000 fieldstats_update_maxperiod=60 bucket=300 extra_field=* | noop 09-01-2023 15:11:38.708 INFO TimelineCreator [1792856 StatusEnforcerThread] - Creating timeline with remote=0 partialCommits=0 commitFreq=0 syncKSFreq=0 maxSyncKSPeriodTime=60000 bucket=300 latestTime=1693573896.000000 earliestTime=1693486800.000000 09-01-2023 15:11:38.713 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - Adding SplunkMR jar to classpath ... 09-01-2023 15:11:38.717 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - HADOOP_USER_CLASSPATH_FIRST= 09-01-2023 15:11:38.717 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - HADOOP_CLASSPATH=:/opt/splunk/bin/jars/SplunkMR-hy3.jar:/opt/splunk/bin/jars/thirdparty/common/avro-1.9.1.jar:/opt/splunk/bin/jars/thirdparty/common/libfb303-0.9.2.jar:/opt/splunk/bin/jars/thirdparty/common/log4j-api-2.17.2.jar:/opt/splunk/bin/jars/thirdparty/common/commons-io-2.4.jar:/opt/splunk/bin/jars/thirdparty/common/log4j-core-2.17.2.jar:/opt/splunk/bin/jars/thirdparty/common/commons-compress-1.21.jar:/opt/splunk/bin/jars/thirdparty/common/parquet-hive-bundle-1.11.2.jar:/opt/splunk/bin/jars/thirdparty/common/avro-mapred-1.9.1.jar:/opt/splunk/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar:/opt/splunk/bin/jars/thirdparty/aws/joda-time-2.8.1.jar:/opt/splunk/bin/jars/thirdparty/aws/jackson-databind-2.13.5.jar:/opt/splunk/bin/jars/thirdparty/aws/httpcore-4.3.3.jar:/opt/splunk/bin/jars/thirdparty/aws/jackson-annotations-2.9.9.jar:/opt/splunk/bin/jars/thirdparty/aws/jackson-core-2.9.9.jar:/opt/splunk/bin/jars/thirdparty/aws/aws-java-sdk-1.10.8.jar:/opt/splunk/bin/jars/thirdparty/aws/httpclient-4.5.13.jar:/opt/splunk/bin/jars/thirdparty/aws/commons-codec-1.15.jar:/opt/splunk/bin/jars/thirdparty/hive_3_1/hive-exec-3.1.3.jar:/opt/splunk/bin/jars/thirdparty/hive_3_1/hive-serde-3.1.3.jar:/opt/splunk/bin/jars/thirdparty/hive_3_1/hive-metastore-3.1.3.jar:/opt/splunk/bin/jars/SplunkMR-h3.jar:/opt/splunk/bin/jars/SplunkMR-hy3.jar 09-01-2023 15:11:38.717 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - Invoking command: /opt/hadoop/bin/hadoop com.splunk.mr.SplunkMR 09-01-2023 15:11:38.890 INFO ReducePhaseExecutor [1792856 StatusEnforcerThread] - ReducePhaseExecutor=1 action=PREVIEW 09-01-2023 15:11:39.648 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR - starting, version=6.2 ... 09-01-2023 15:11:39.957 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - 2023-09-01 14:11:39,957 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS 09-01-2023 15:11:39.961 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR - Setting custom jars=file:/opt/splunk/bin/jars/SplunkMR-hy3.jar,file:/opt/splunk/bin/jars/thirdparty/common/avro-1.9.1.jar,file:/opt/splunk/bin/jars/thirdparty/common/libfb303-0.9.2.jar,file:/opt/splunk/bin/jars/thirdparty/common/log4j-api-2.17.2.jar,file:/opt/splunk/bin/jars/thirdparty/common/commons-io-2.4.jar,file:/opt/splunk/bin/jars/thirdparty/common/log4j-core-2.17.2.jar,file:/opt/splunk/bin/jars/thirdparty/common/commons-compress-1.21.jar,file:/opt/splunk/bin/jars/thirdparty/common/parquet-hive-bundle-1.11.2.jar,file:/opt/splunk/bin/jars/thirdparty/common/avro-mapred-1.9.1.jar,file:/opt/splunk/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,file:/opt/splunk/bin/jars/thirdparty/aws/joda-time-2.8.1.jar,file:/opt/splunk/bin/jars/thirdparty/aws/jackson-databind-2.13.5.jar,file:/opt/splunk/bin/jars/thirdparty/aws/httpcore-4.3.3.jar,file:/opt/splunk/bin/jars/thirdparty/aws/jackson-annotations-2.9.9.jar,file:/opt/splunk/bin/jars/thirdparty/aws/jackson-core-2.9.9.jar,file:/opt/splunk/bin/jars/thirdparty/aws/aws-java-sdk-1.10.8.jar,file:/opt/splunk/bin/jars/thirdparty/aws/httpclient-4.5.13.jar,file:/opt/splunk/bin/jars/thirdparty/aws/commons-codec-1.15.jar,file:/opt/splunk/bin/jars/thirdparty/hive_3_1/hive-exec-3.1.3.jar,file:/opt/splunk/bin/jars/thirdparty/hive_3_1/hive-serde-3.1.3.jar,file:/opt/splunk/bin/jars/thirdparty/hive_3_1/hive-metastore-3.1.3.jar,file:/opt/splunk/bin/jars/SplunkMR-h3.jar 09-01-2023 15:11:40.003 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex$VIXPathSpecifier - VIXPathSpecifier globpath=/indexes/*, accept=null, ignore=null 09-01-2023 15:11:40.007 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkBaseMapper - RecordReader list, index=null, inputId=null, list=SplunkLineRecordReader 09-01-2023 15:11:40.013 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR$SearchHandler - Reduce search: null 09-01-2023 15:11:40.013 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR$SearchHandler - Search mode: stream 09-01-2023 15:11:40.013 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR$SearchHandler - parsed _keySet=[index::hadoop] 09-01-2023 15:11:40.013 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR$SearchHandler - setting requiredFields=* 09-01-2023 15:11:40.683 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR$SearchHandler - Created filesystem object, elapsed_ms=670 09-01-2023 15:11:40.760 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SSLUtil$SSLv3SocketFactory - Changing enabled SSL protocols from ['TLSv1.3', 'TLSv1.2'] to ['TLSv1.1', 'TLSv1', 'TLSv1.3', 'TLSv1.2'] 09-01-2023 15:11:40.874 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - 2023-09-01 14:11:40,874 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 09-01-2023 15:11:40.874 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SearchController - stopOnMaxHadoopNodesExceedingLicense ... 09-01-2023 15:11:40.883 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VixSplitGenerator - Generating splits for index=hadoop 09-01-2023 15:11:40.886 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex$Splitter - generateSplits started, vix.name=hadoop ... 09-01-2023 15:11:40.889 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex$Splitter - using SplitGenerator class=com.splunk.mr.input.FileSplitGenerator, to split files in index=hadoop inputId=1 09-01-2023 15:11:40.943 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - 2023-09-01 14:11:40,942 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /192.168.100.72:8032 09-01-2023 15:11:40.975 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex$VIXPathSpecifier - readFromDM=false, writeToDM=false 09-01-2023 15:11:41.020 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex$VIXPathSpecifier - working on path=hdfs://192.168.100.72:9000/indexes/dnf.rpm.log, mtime=1693569669200, isDir=false 09-01-2023 15:11:41.020 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex$VIXBucketSpecifier - Got bucket id for path: _no_id 09-01-2023 15:11:41.020 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex - File meets time heuristic path=hdfs://192.168.100.72:9000/indexes/dnf.rpm.log, search.et=1693486800, search.lt=1693573896, file.et=0, file.lt=9223372036854775807, file.mtime=1693569669 09-01-2023 15:11:41.021 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - 2023-09-01 14:11:41,021 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS 09-01-2023 15:11:41.021 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex - File meets the search criteria. Will consider it, path=hdfs://192.168.100.72:9000/indexes/dnf.rpm.log 09-01-2023 15:11:41.030 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - BaseSplunkRecordReader - requiredFields=* 09-01-2023 15:11:41.134 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - JobSubmitterInputFormat - createRecordReader: /indexes/dnf.rpm.log:0+134217728 09-01-2023 15:11:41.136 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkBaseMapper - RecordReader list, index=hadoop, inputId=1, list=SplunkLineRecordReader 09-01-2023 15:11:41.138 DEBUG ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - BaseSplunkRecordReader - requiredFields=* 09-01-2023 15:11:41.157 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - JobSubmitterInputFormat - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=/indexes/dnf.rpm.log:0+134217728 09-01-2023 15:11:41.157 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=/indexes/dnf.rpm.log:0+134217728 09-01-2023 15:11:41.221 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - 2023-09-01 14:11:41,221 WARN mapred.ResourceMgrDelegate: getBlacklistedTrackers - Not implemented yet 09-01-2023 15:11:41.222 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - ClusterInfoLogger - Hadoop cluster spec: provider=MyHadoopProvider, tasktrackers=1, map_inuse=1, map_slots=10, reduce_inuse=1, reduce_slots=2 09-01-2023 15:11:41.264 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - VirtualIndex - generateSplits done, vix.name=hadoop, files.total=1, files.time.filtered=0, files.search.filtered=0, files.mr=1, elapsed=380ms 09-01-2023 15:11:41.266 INFO ERP.MyHadoopProvider [1792956 StreamScriptThreadfor/opt/hadoop/bin/hadoop] - SplunkMR - finishing, version=6.2 ...