Can we run IDL(Interactive Data Language) in Databricks Notebook (python code)

Question

I am working on a project where we use Azure stack for data engineering and analysis. The main component for computation is Azure Databricks in which most of the code is written in python code.

Recently I got a requirement to work in a project where we have to process mf4(Measurement Data Files) files. To process mf4 files, we sort out the solution to use asammdf library and process the file.

Then the next phase came to migrate few formulas which are in IDL(Interactive Data Language). These formulas are stored in an Oracle database and we are able to connect to the database in order to get the formulas.

But the question arise here is How to run these IDL formulas on the data available in Azure Storage. Can we run these formulas through Python notebook file and use the data for analysis.

I have gone through few documentations and didn't get any Idea to implement a solution for this.

Is it possible to import any IDL library in python notebook files and execute IDL formulas ?
Is this IDL an open source or licensed product?

Any leads Appreciated! Thanks in Advance.

Tried below command to import the module, but not working.from idlpy import * — Antony
– Antony, Commented Nov 20, 2023 at 10:08

Jaya Shankar G S · Accepted Answer · 2023-11-29 09:01:22Z

0

First of all, the product is licensed. After obtaining a license, follow the steps below.

Install the IDL from the steps provided in this link.

Before doing that, you should have downloaded the IDL .tar.gz. Download it by creating an account and downloading it from the product download page.

enter image description here

Next, upload it to the DBFS in Databricks and start the installation.

After extracting, run the command below since Databricks doesn't support prompt input.

echo -e "y\n/usr/local/harris\ny\nUnix\ny\n" | ./install.sh -s

Your installation directory will be /usr/local/harris.

Next, you need to activate this with the activation code you received earlier.

Then, according to this documentation, set up the path and import.

import sys
sys.path.append('/usr/local/harris/idl/lib/bridges')
from idlpy import  *

Here, your installation directory is /usr/local/harris. Provide that.

If you want to use the command line, do

%sh
idl

enter image description here

Since I do not have an activation code, it is showing to activate it.

answered Nov 29, 2023 at 9:01

Jaya Shankar G S

8,6282 gold badges6 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Antony Over a year ago

Is there any open source library for python which is compatible with IDL ?

Jaya Shankar G S Over a year ago

pypi.org/project/idlmagic. Works only if you have idl installed

Collectives™ on Stack Overflow

Can we run IDL(Interactive Data Language) in Databricks Notebook (python code)

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related