3

I want to use a web scraping python script in my batch program but I cannot find any examples in the web for this. Hence, it's possible to use python scripts with the ScriptItemProcessor?

Here's my simple code to test this:

@Bean
public ScriptItemProcessor<Company,Company> scriptItemProcessor() {
    return new ScriptItemProcessorBuilder<Company,Company>().language("python").scriptSource("print('hello')").build();
}

UPDATE 1

After adding the required jar the exception changed, so now is recognizing it but the new stack trace is not very descriptive at all.

    java.lang.IllegalStateException: Script engine with name 'python' failed to initialize
        at org.springframework.scripting.support.StandardScriptUtils.retrieveEngineByName(StandardScriptUtils.java:64) ~[spring-context-5.3.3.jar!/:5.3.3]
        at org.springframework.scripting.support.StandardScriptEvaluator.getScriptEngine(StandardScriptEvaluator.java:174) ~[spring-context-5.3.3.jar!/:5.3.3]
        at org.springframework.scripting.support.StandardScriptEvaluator.evaluate(StandardScriptEvaluator.java:143) ~[spring-context-5.3.3.jar!/:5.3.3]
        at org.springframework.batch.item.support.ScriptItemProcessor.process(ScriptItemProcessor.java:64) ~[spring-batch-infrastructure-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.doProcess(SimpleChunkProcessor.java:134) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.transform(SimpleChunkProcessor.java:319) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:210) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:77) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:407) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.3.3.jar!/:5.3.3]
        at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:258) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:137) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:147) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.3.3.jar!/:5.3.3]
        at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.3.jar!/:5.3.3]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.3.jar!/:5.3.3]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.3.jar!/:5.3.3]
        at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128) ~[spring-batch-core-4.3.1.jar!/:4.3.1]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.3.jar!/:5.3.3]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.3.jar!/:5.3.3]
        at com.sun.proxy.$Proxy73.run(Unknown Source) ~[na:na]
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199) ~[spring-boot-autoconfigure-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173) ~[spring-boot-autoconfigure-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160) ~[spring-boot-autoconfigure-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155) ~[spring-boot-autoconfigure-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150) ~[spring-boot-autoconfigure-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:795) ~[spring-boot-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:785) ~[spring-boot-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:333) ~[spring-boot-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1311) ~[spring-boot-2.4.2.jar!/:2.4.2]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) ~[spring-boot-2.4.2.jar!/:2.4.2]
        at com.stockanalyzer.stockanalyzer.StockAnalyzerApplication.main(StockAnalyzerApplication.java:136) ~[classes!/:0.0.1-SNAPSHOT]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[stock-analyzer-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:107) ~[stock-analyzer-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[stock-analyzer-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) ~[stock-analyzer-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
Caused by: java.lang.NullPointerException: null
        at org.python.core.PySystemState.<init>(PySystemState.java:214) ~[jython-slim-2.7.2.jar!/:na]
        at org.python.util.PythonInterpreter.threadLocalStateInterpreter(PythonInterpreter.java:80) ~[jython-slim-2.7.2.jar!/:na]
        at org.python.jsr223.PyScriptEngine.<init>(PyScriptEngine.java:27) ~[jython-slim-2.7.2.jar!/:na]
        at org.python.jsr223.PyScriptEngineFactory.getScriptEngine(PyScriptEngineFactory.java:85) ~[jython-slim-2.7.2.jar!/:na]
        at org.springframework.scripting.support.StandardScriptUtils.retrieveEngineByName(StandardScriptUtils.java:60) ~[spring-context-5.3.3.jar!/:5.3.3]
        ... 57 common frames omitted

Final Update

I made it work by adding this dependency in the pom.xml...

        <dependency>
            <groupId>org.python</groupId>
            <artifactId>jython-slim</artifactId>
            <version>2.7.2</version>
        </dependency>

... and added what's inside the requiresUnpack tag:

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <configuration>
                <excludes>
                    <exclude>
                        <groupId>org.projectlombok</groupId>
                        <artifactId>lombok</artifactId>
                    </exclude>
                </excludes>
                <requiresUnpack>
                    <dependency>
                        <groupId>org.python</groupId>
                        <artifactId>jython-slim</artifactId>
                    </dependency>
                </requiresUnpack>
            </configuration>
        </plugin>
    </plugins>
</build>

PD: remember the item processor needs to be configured in the StepBuilder.

Improve this question

1 Answer 1

Reset to default
3

The ScriptItemProcessor delegates item processing to a org.springframework.scripting.ScriptEvaluator. As of v5.3, Spring Framework provides three implementations of the ScriptEvaluator interface: BshScriptEvaluator, GroovyScriptEvaluator and StandardScriptEvaluator.

By default, if you don't specify which ScriptEvaluator to use, the ScriptItemProcessor will use a StandardScriptEvaluator, see Javadoc of setScriptEvaluator. Now this StandardScriptEvaluator is based on JSR-223 javax.script, so you need to make sure the language you want to use is supported by the default ScriptEngine. In your case, python is not a supported language by default, and hence the error.

If you want to use python through JSR-223, you can use Jython (adding the jar of Jython to the classpath should be enough, see Calling Python from Java through scripting engine (jython)?)

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thank you @Mahmoud Ben Hassine I still have a exception but know is different and the stack trace shows that the jar is recognized. I'm going to edit the post to show the new exception.
Based on your new stack trace, this is a NullPointerException in Jython code: Caused by: java.lang.NullPointerException: null at org.python.core.PySystemState.<init>(PySystemState.java:214) ~[jython-slim-2.7.2.jar!/:na]. So you need to open an issue against Jython.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.