Hello everyone! I'm using Laravel Excel to handle exports and imports with more than 300K records, but I'm facing memory usage and timeout issues. I'm already using queued exports, but large datasets still cause problems. What are the best practices to optimize performance and prevent memory exhaustion? thank you!
#I'm using Laravel Excel to handle exports and imports with more than 300K records, but I'm facing me
faint jolt
high notch
You'd need to use chunked reading and maybe even batched inserts;
https://docs.laravel-excel.com/3.1/imports/chunk-reading.html
https://docs.laravel-excel.com/3.1/imports/batch-inserts.html
The thing is, you're loading a whole lot at once into memory, so by using chunked reading you're only reading/processing part of the file.
faint jolt
Really Appreciate Sir!! and is the same as export?
high notch
An export is slightly different, but you could use an iterator along with a cursor (I think) https://laravel.com/docs/12.x/eloquent#cursors
Or use a chunked query. Kinda depends on your needs
faint jolt
Sorry for the late Sir, I was already using chunks "public function chunkSize(): int
{
return 5000;
}" ```php class StudentExport implements FromQuery, ShouldAutoSize, ShouldQueue, WithChunkReading, WithColumnWidths, WithHeadings, WithStyles
{
use Exportable;
public function query()
{
return Employee::select('full_name','emp_id');
}
public function headings(): array
{
return [
'Full Name',
'Employee ID',
];
}
public function columnWidths(): array
{
return [
'A' => 30,
'B' => 30,
];
}
public function styles(Worksheet $sheet)
{
return [
1 => [
'font' => ['bold' => true],
],
'C' => ['alignment' => ['horizontal' => 'center']],
];
}
public function chunkSize(): int
{
return 5000;
}
}```
bronze trench
This might help: https://www.youtube.com/watch?v=CAi4WEKOT4A
faint jolt
let me check again! thanks a lot Sir