0

Is it possible to refactor this script such that it exists as a completely independent method?

import json
import requests
from collections import defaultdict
from pprint import pprint

def hasNumbers(inputString):
    return any(char.isdigit() for char in inputString)

# open up the output of 'data-processing.py'
with open('job-numbers-by-location.txt') as data_file:

    # print the output to a file
    with open('phase_ii_output.txt', 'w') as output_file_:
        for line in data_file:
            identifier, name, coords, number_of_jobs = line.split("|")
            coords = coords[1:-1]
            lat, lng = coords.split(",")
            # print("lat: " + lat, "lng: " + lng)
            response = requests.get("http://api.geonames.org/countrySubdivisionJSON?lat="+lat+"&lng="+lng+"&username=s.matthew.english").json()


            codes = response.get('codes', [])
            for code in codes:
                if code.get('type') == 'ISO3166-2':
                    country_code = '{}-{}'.format(response.get('countryCode', 'UNKNOWN'), code.get('code', 'UNKNOWN'))
                    if not hasNumbers( country_code ):
                        # print("code: " + country_code + ", jobs: " + number_of_jobs)
                        output_file_.write("code: " + country_code + ", jobs: " + number_of_jobs)
    output_file_.close()

I've been trying to make it so that I can include it as a component of a much larger process.

1 Answer 1

2

There are a couple of things you can do. You might want to break each step of the script into separate methods each with their own exception handling, and logging to indicate where the job has failed. Also, I have not mentioned the return parameter here. You can return a True/False to suggest whether the processing has passed/failed.

You can then import the process_file method in other places, and pass it the 2 files that it needs to process.

import json
import requests
from collections import defaultdict
from pprint import pprint

def hasNumbers(inputString):
    return any(char.isdigit() for char in inputString)

def handle_get(url, params)
    try:
        response = requests.get(url, params=urlencode(params))
    except requests.exceptions.RequestException as e:  # This is the correct syntax
        print e
        # sys.exit(1)
        response = None

    return response

def process_file(data_file_path, output_file_path)
    # open up the output of 'data-processing.py'
    with open(data_file_path) as data_file:

        # print the output to a file
        with open(output_file_path, 'w') as output_file_:
            for line in data_file:
                identifier, name, coords, number_of_jobs = line.split("|")
                coords = coords[1:-1]
                lat, lng = coords.split(",")
                params = OrderedDict([('lat', lat), ('lng', lng), ('username', 's.matthew.english')])
                url = "http://api.geonames.org/countrySubdivisionJSON"
                response = handle_get(url, params)
                if response:
                    json_response = response.json()
                else:
                    print('Something bad happened')
                    sys.exit(1)


                codes = response.get('codes', [])
                for code in codes:
                    if code.get('type') == 'ISO3166-2':
                        country_code = '{}-{}'.format(response.get('countryCode', 'UNKNOWN'), code.get('code', 'UNKNOWN'))
                        if not hasNumbers( country_code ):
                            # print("code: " + country_code + ", jobs: " + number_of_jobs)
                            output_file_.write("code: " + country_code + ", jobs: " + number_of_jobs)
Sign up to request clarification or add additional context in comments.

1 Comment

I'm trying to incorporate it eventually into this thing

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.