Remove duplicate records from csv while importing in php

Question

I want to import a csv file from a html form and process it in php. Dynamically create a table and append it to my div for preview purposes.

I want array only of unique records inside the table.

I have following code in php:

$tmpName = $_FILES['blk_csv']['tmp_name'];

$csvAsArray = array_unique((array_map('str_getcsv', file($tmpName))),SORT_REGULAR);

$table_data = '<table id="template_table" name="template_table" class="table">';
$csvAsArrayU = $csvAsArray;

for ($i = 0; $i < count($csvAsArrayU); $i++) {
     
     if($i == 0){
        
         $table_data .= '<thead><tr>';
        
         for($h = 0 ; $h < count($csvAsArrayU[$i]); $h++){
            
            
             $table_data .= '<th class="text-center">'.$csvAsArrayU[$i][$h].'</th>';

         }
        
         $table_data .= '</tr></thead><tbody>';

     }
      else{
         $table_data .= '<tr id="row-'.$i.'">';
            
         for($r = 0 ; $r < count($csvAsArrayU[$i]); $r++){
                
            $table_data .= '<td  class="text-center">'.$csvAsArrayU[$i][$r].'</td>';

         }
         $table_data .= '</tr>';
        
      }
    
}
$table_data .= ' </tbody></table>';

My csv structure

 NAME,EMAIL
 Morris,[email protected]
 Morris,[email protected]
 Morris,[email protected]
 Jacob,[email protected]

If you print_r($csvAsArray) it gives me following output:

Array ( [0] => Array ( [0] => NAME [1] => EMAIL ) [1] => Array ( [0] => Morris [1] => [email protected] ) [8] => Array ( [0] => Jacob [1] => [email protected] ) )

If you notice the index for third element is 8 and not 2. Thus the DataTable library fails to load the data and shows something like this :

Can any one guide me how to fix this ? Or may be a different approach to load only unique rows in array ?

Your approach is fine. Note that array_unique preserves the keys. The entry in question is presumably in that position in the source CSV, simple as that. You could also run array_unique on file and only map the remaining lines for CSV. As it stands, you're iterating a lot of duplicates that are then discarded. That would also give you consecutive indices without having to run array_values. — Markus AO
– Markus AO, Commented Oct 24, 2020 at 13:19
You could also really just use a foreach loop, rather than accessing the array with a count-based for loop. That's what they're meant for. — Markus AO
– Markus AO, Commented Oct 24, 2020 at 13:29
can you please provide a example ? I would be very great full to understand a different approach. — Vikrant
– Vikrant, Commented Oct 24, 2020 at 13:55
I've rewritten the relevant parts and posted it as an answer. — Markus AO
– Markus AO, Commented Oct 24, 2020 at 14:56

Manav · Accepted Answer · 2020-10-24 13:04:47Z

1

$csvAsArray = array_values($csvAsArray)

should work for you, this would reset the keys of the array, so that the index of the third element is 2.

:)

answered Oct 24, 2020 at 13:04

Manav

1,36710 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Vikrant Over a year ago

Well, that small line of code has done the magic. Proves my research is poor. Thanks for helping me !

Markus AO · Accepted Answer · 2020-10-24 15:00:56Z

OP asked for a different approach in the comments. Here's a rewrite that implements my suggestions. I've fluffed out function cascades into one-function-per-line to keep this more educational. Please read the comments, understand the functions used, and remember: the manual is your best friend. Now, the code:

$tmpName = 'tmp/file.csv';

// Read lines; Filter for unique lines; Map array as CSV:
$rawLines = file($tmpName, FILE_IGNORE_NEW_LINES|FILE_SKIP_EMPTY_LINES);
$uniqLines = array_unique($rawLines);
$csvAsArray = array_map('str_getcsv', $uniqLines);

// Get the header row:
$csvHeaderRow = array_shift($csvAsArray);

// Build table header:
$table_data = '<table id="template_table" name="template_table" class="table">';
$table_data .= '<thead><tr>';

// Iterate each header field:
foreach($csvHeaderRow as $field){
    $table_data .= '<th class="text-center">'.$field.'</th>';
}

$table_data .= '</tr></thead><tbody>';

// Build table body, iterate lines/rows:
foreach($csvAsArray as $i => $row){
    $table_data .= '<tr id="row-'.$i.'">';

    // Iterate each field in the row:
    foreach($row as $field) {
        $table_data .= '<td  class="text-center">'.$field.'</td>';
    }

    $table_data .= '</tr>';
}

// Wrap up table
$table_data .= ' </tbody></table>';

Above code follows OP's code as far as possible. We use appropriate flags for file to remove clutter in the beginning. We filter the array of strings for unique values, rather than parsing everything as CSV and comparing arrays (as in OP), to avoid redundant processing. We use a foreach loop that's agnostic to having consecutive indices, and the typical way of iterating an array. This results in the following output (without newlines, hard to read):

<table id="template_table" name="template_table" class="table"><thead><tr><th class="text-center">NAME</th><th class="text-center">EMAIL</th></tr></thead><tbody><tr id="row-0"><td  class="text-center">Morris</td><td  class="text-center">[email protected]</td></tr><tr id="row-1"><td  class="text-center">Jacob</td><td  class="text-center">[email protected]</td></tr> </tbody></table>

If you wanted an easy way to add newlines and make your HTML output more readable. And would rather not tack a "\n" at the end of each line (ugly!). Then, instead of string concatenation ($table_data .=), you can append to an array ($table_data[] = ), and in the end implode("\n", $table_data), and your HTML output will be much more readable.

Collectives™ on Stack Overflow

Remove duplicate records from csv while importing in php

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related