How to open multiple Json files and save them in multiple variables

Question

I have some Json files. The naming convention of the file is dataset_ML-Model_params.json. For example, House_Lasso_params.json, House_RF_params.json, Bike_Lasso_params.json, and Bike_RF_params.json.

All of these files contains tuning-hyperparameters in dict format. I can open 1 file using the below code

filename = f"{args.dataset}_Lasso_params.json"
    outfile = HT_OUT / filename
    with open(outfile, "r") as file:
        d_loaded = json.load(file)

Passing the value to the model.

Lasso(**d_loaded, precompute=True)

Again for another file

filename = f"{args.dataset}_RF_params.json"
    outfile = HT_OUT / filename
    with open(outfile, "r") as file:
        rf_loaded = json.load(file)

RF(**rf_loaded)

Here, args.dataset contains the dataset name. Could you tell me, how can I load these 2 files and save them in different variables. So that later i can pass the variable to the model. Like

# After opening and saving the json file in different variable
Lasso(**lasso_params, precompute=True)
RF(**rf_params)

Why not simple move the code of passing the the parameters to the values after you've read all files? Or are you seeking a more generic solution when you have a lot of files? — NLindros
– NLindros, Commented Jul 14, 2021 at 15:07
Thanks. Yes, I have a lot of files. So, I wanted some pythonic way. — Opps_0
– Opps_0, Commented Jul 14, 2021 at 15:19

0Knowledge · Accepted Answer · 2021-07-14 15:18:15Z

1

Make a list of all models

MODEL_NAMES = ["Lasso", "Ridge"]

Make another dictionary to save the params for each model

    models_params = {}
    for model_name in MODEL_NAMES:
        filename = f"{args.dataset}_{model_name}_params.json"
        outfile = HT_OUT / filename
        with open(outfile, "r") as file:
            d_loaded = json.load(file)
            models_params[model_name] = d_loaded

Later, use the get(key) to access your expected params.

Lasso(**(models_params.get('Lasso')), precompute=True)
RF(**(models_params.get('RF')))

You can also check the params

print(Lasso(**(models_params.get('Lasso')), precompute=True).get_params())

answered Jul 14, 2021 at 15:18

0Knowledge

7651 gold badge6 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

maevgor · Accepted Answer · 2021-07-14 15:07:18Z

1

You could use another dict that gonna contain params that you need. For example,

model_params = {'lasso_params': smth_here, 'rf_params': smth_here}

So then you can get needed value by

*model_params['lasso_params']

To get all files by that wildcard (dataset_ML-Model_params.json.) you could use library called glob:

from glob import glob

glob('*_params.json') # return ['lasso_params', 'rf_params', ...]

And then just read them one by one.

answered Jul 14, 2021 at 15:07

maevgor

562 bronze badges

Collectives™ on Stack Overflow

How to open multiple Json files and save them in multiple variables

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related