0

I'm getting different responses depending on wether I run the query in an "all in one" query or if I run them separately.

I can't for the life of me figure out what I'm doing wrong.

Code:

import mysql.connector


def all_in_one():
    try:
        conn = mysql.connector.connect(
            host="localhost", user="fanw", password="secret", database="fanw"
        )

        cursor = conn.cursor()

        query = """
            CREATE TEMPORARY TABLE IF NOT EXISTS temp_pairs AS

            SELECT
                fd.nid,
                fd.title
            FROM
                node__field_card_display cl
            JOIN 
                node_field_data fd ON cl.entity_id = fd.nid AND cl.langcode = fd.langcode
            WHERE
                cl.field_card_display_target_id = 2049
                AND cl.langcode = 'en'
                AND cl.bundle = 'person';

            SELECT
                t.entity_id
            FROM
                node__body AS t
            JOIN
                temp_pairs AS tp ON BINARY t.body_value LIKE CONCAT('%', tp.title, '%') 
                AND NOT (t.body_value LIKE CONCAT('%', tp.nid, '">', tp.title, '<', '%'))
            WHERE
                t.body_format <> 'plain_text'
                AND t.entity_id <> tp.nid;
        """
        for each in cursor.execute(query, multi=True):
            result = each.fetchall()
            print(len(result))

        conn.commit()

    except Exception as e:
        # Handle any errors and potentially roll back the transaction
        conn.rollback()
        print(f"Error: {str(e)}")

    finally:
        cursor.close()
        conn.close()


def separate():
    try:
        conn = mysql.connector.connect(
            host="localhost", user="fanw", password="secret", database="fanw"
        )
        cursor = conn.cursor()

        query = """
            SELECT
                fd.nid,
                fd.title
            FROM
                node__field_card_display cl
            JOIN 
                node_field_data fd ON cl.entity_id = fd.nid AND cl.langcode = fd.langcode
            WHERE
                cl.field_card_display_target_id = 2049
                AND cl.langcode = 'en'
                AND cl.bundle = 'person';
        """
        cursor.execute(query)
        result = cursor.fetchall()
        people = []
        for person in result:
            nid, name = person
            people.append((nid, name))

        cursor.execute(
            "CREATE TEMPORARY TABLE IF NOT EXISTS temp_pairs (nid INT(10) unsigned, title VARCHAR(255));"
        )
        cursor.executemany(
            "INSERT INTO temp_pairs (nid, title) VALUES (%s, %s);", people
        )

        query = f"""
            SELECT
                t.entity_id
            FROM
                node__body AS t
            JOIN
                temp_pairs AS tp ON BINARY t.body_value LIKE CONCAT('%', tp.title, '%') 
                AND NOT (t.body_value LIKE CONCAT('%', tp.nid, '">', tp.title, '<', '%'))
            WHERE
                t.body_format <> 'plain_text'
                AND t.entity_id <> tp.nid;
        """
        cursor.execute(query)
        result = cursor.fetchall()
        print(len(result))

        conn.commit()

    except Exception as e:
        # Handle any errors and potentially roll back the transaction
        conn.rollback()
        print(f"Error: {str(e)}")

    finally:
        cursor.close()
        conn.close()


if __name__ == "__main__":
    print("All in one: ")
    all_in_one()

    print("Separate:")
    separate()
    print("DONE!")

Running the code gives:

❯ python3 migrate.py                     
All in one: 
0
96
Separate:
0
DONE!

I would expect the "separate" query to also give 96

I've tried both mysql-connector-python==8.3.0 and PyMySQL==1.1.0.

I've run the "all in one" query in both the mysql CLI and using DBeaver and both gives me 96

I'm running this on a MacBook Pro M1, but given the "all in one" works, I don't think that is relevant.

4
  • You didn't conn.commit() your entries in the "separate" query before running the next command. Why that's not an error when you SELECT after, I'm not sure, but I'd bet money on it being the lack of commit. I don't use MySQL so I'm not sure about transaction isolation here, but you don't appear to be trying to use it and I wouldn't be surprised if that was the issue Commented Apr 8, 2024 at 18:58
  • @roganjosh I sprinkled with commits and also added this select to the last separate query: ` SELECT count(*) FROM temp_pairs;` and it gave me the expected value. :( Commented Apr 8, 2024 at 20:02
  • I'm not sure why that's a bad thing or why you didn't expect that result Commented Apr 8, 2024 at 20:11
  • @roganjosh Sorry, I should've been clearer. The select gives med 448 which are the number of entries in the temp_pairs table, but the select that matters still gives me 0 - while I expect 96. Commented Apr 8, 2024 at 20:30

1 Answer 1

1

As usual, I'll answer my own question.

tl;dr: it was the character set (and potentially the collation).

So when you do “CREATE TEMPORARY TABLE … AS” it will inherit those settings from the SELECT statement.

In this case utf8mb4 (and utf8mb4_general_ci).

But if you do a “standalone” “CREATE TEMPORARY TABLE” (as in the separate function), the table inherits the database default, which in this case was latin1 and since the temp_pairs table contains names of people from all over the world, if they had a character in their name that was outside of the latin1 charset, it wasn’t picked up by the query.

So the correct query would be:

CREATE TEMPORARY TABLE IF NOT EXISTS
temp_pairs
(
  nid INT(10) unsigned NOT NULL, title VARCHAR(255) NOT NULL
) ENGINE=InnoDB CHARACTER SET 'utf8mb4' COLLATE 'utf8mb4_general_ci';

As a sidenote, the for person in result-loop is redundant as the result is already a list of tuples.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.