🔒 JSON object not loading as expected

[quick hare]

    # Get the address metadata
    response = await address_search(address=address, results_limit=1)
    response_body_str = response.body.decode()
    print(response_body_str)
    
    address_search_results = json.loads(response_body_str)

    print(address_search_results)
    print(type(address_search_results))

    #Extract the Latitude and Longitude from the address search results
    latitude = float(address_search_results['features'][0]['properties']['LATITUDE'])
    longitude = float(address_search_results['features'][0]['properties']['LONGITUDE'])

This is returning the following error:

  File "/code/routers/address_and_meta_search.py", line 29, in single_address_and_meta_search
    latitude = address_search_results['features'][0]['properties']['LATITUDE']
               ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
TypeError: string indices must be integers, not 'str'

But when I replicate it in a terminal, it works fine?

response_body_str = "{\"type\": \"FeatureCollection\", \"features\": [{\"id\": \"0\", \"type\": \"Feature\", \"properties\": {\"LATITUDE\": \"-37.00000\", \"LONGITUDE\": \"145.00000\", \"@search.score\": 8.628523, \"@search.reranker_score\": null, \"@search.highlights\": null, \"@search.captions\": null}, \"geometry\": {\"type\": \"Point\", \"coordinates\": [145.0000, -37.00000]}}]}"

[steel tide]

@quick hare

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[quick hare]

response_body_str = "{\"type\": \"FeatureCollection\", \"features\": [{\"id\": \"0\", \"type\": \"Feature\", \"properties\": {\"STREET_TYPE\": \"AVENUE\", \"STREET_NAME\": \"NARCISSUS\", \"FLAT_NUMBER\": \"NULL\", \"POSTCODE\": \"3155\", \"LATITUDE\": \"-37.86247881\", \"BUILDING_NAME\": \"NULL\", \"SUBURB\": \"BORONIA\", \"LONGITUDE\": \"145.27822418\", \"LEVEL_NUMBER\": \"nan\", \"index\": \"248943\", \"NUMBER_FIRST\": \"1\", \"FLAT_TYPE\": \"NULL\", \"ADDRESS\": \"1 NARCISSUS AV, BORONIA VIC 3155\", \"NUMBER_LAST\": \"nan\", \"STATE\": \"VIC\", \"@search.score\": 8.628523, \"@search.reranker_score\": null, \"@search.highlights\": null, \"@search.captions\": null}, \"geometry\": {\"type\": \"Point\", \"coordinates\": [145.27822418, -37.86247881]}}]}"
address_search_results = json.loads(response_body_str)
latitude = float(address_search_results['features'][0]['properties']['LATITUDE'])

[stable crest]

I agree that should be working.
What are you getting in your debug prints above the indexing?

[quick hare]

https://pastebin.com/aWjUkqC8

I'm running this as a FastAPI call

[distant heath]

what are these printing

    print(address_search_results)
    print(type(address_search_results))

[stable crest]

Yeah, this is the crux of it.
It musn't be the json you expect it to be.

[quick hare]

{"type": "FeatureCollection", "features": [{"id": "0", "type": "Feature", "properties": {"STREET_NAME": "NARCISSUS", "LATITUDE": "-37.86247881", "LEVEL_NUMBER": "nan", "STATE": "VIC", "POSTCODE": "3155", "ADDRESS": "1 NARCISSUS AV, BORONIA VIC 3155", "STREET_TYPE": "AVENUE", "BUILDING_NAME": "NULL", "index": "248943", "FLAT_TYPE": "NULL", "NUMBER_LAST": "nan", "SUBURB": "BORONIA", "NUMBER_FIRST": "1", "LONGITUDE": "145.27822418", "FLAT_NUMBER": "NULL", "@search.score": 8.628523, "@search.reranker_score": null, "@search.highlights": null, "@search.captions": null}, "geometry": {"type": "Point", "coordinates": [145.27822418, -37.86247881]}}]}
<class 'str'>

[ripe laurel]

It looks like the response is being wrapped in an extra set of ""'s, which is causing the JSON to be parsed as a single string. Does this fix it?:

address_search_results = json.loads(json.loads(response_body_str))

[quick hare]

Yes it does - I take it somewhere, somehow it's getting encoded twice?

[ripe laurel]

Lol

I'm not sure

[quick hare]

I don't understand how it that occurs!

I can't even see where it's getting wrapped twice, I figured response_body_str is the string representation, then it's getting parsed into json.loads and then voila, it should be a json object.

[distant heath]

it is, it's a JSON string representing a JSON object

[quick hare]

https://tenor.com/view/ohh-oh-damn-oh-daymn-daym-damn-gif-2649905700736798302

[ripe laurel]

I don't know what address_search is, but it's probably to blame.

[quick hare]

It's me. I am to blame.

[ripe laurel]

Or the API that it's using

[distant heath]

!e

import json

print(repr(
    json.dumps(json.dumps(
        {1: 2}
    ))
))

[steel tide]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

'"{\\"1\\": 2}"'

[quick hare]

 Create a FastAPI Endpoint that will do a search in Azure Search of an address, which returns the address and the coordinates for vic_fire_permits_rev_geo
@router.get('/')
async def address_search(address:str, results_limit:int=1)->Response:
    """Takes an address string and returns the address metadata, including Longitude and Latitude.
    # TODO: improve the JSON index and encoding for null values. 
    Args:
        address (str): Address string to search for.
        results_limit (int, optional): Number of results to limit the search to. Defaults to 1. #TODO: Enforce > 0

    Returns:
        Response: JSON response for FastAPI delivery.
    """

    search_client = get_search_client()
    results = search_address_string(search_client, address, results_limit=results_limit)
    result_df = pd.DataFrame(results)
    results_df = gpd.GeoDataFrame(result_df, geometry=gpd.points_from_xy(result_df.LONGITUDE, result_df.LATITUDE)) 
    #return Response(content=results_df.to_json(), media_type='application/json')

    return JSONResponse(content=results_df.to_json(), media_type='application/json')

I take it JSONresponse will encode the Dataframe

Which makes sense, because it trying to troubleshoot an earlier problem, I changed it from Response to JSONResponse

[ripe laurel]

You're internally calling a route? You probably want to be separating the core logic from the web-server specific aspects.

[quick hare]

return Response(content=results.toJSON(), media_type='application/json')

[ripe laurel]

Make a library function that does the core stuff, call that from the endpoint, and then from this code, use the library function. The more "raw" the data you're dealing with, the less confusion there is.

If you're trying to use objects meant to be used as a HTTP return value as a normal value, there may be problems.

[quick hare]

Thank you, I'll go refactor it now.

!close

[steel tide]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.