Airbyte connector not syncing records created after 9/10

[vital nova]

The sync executes successfully right now, but doesn't seem to be syncing any records created after September 10th 🤔

[slow shoal]

Deployment name? I can check up on it to see what's going on

[vital nova]

giant-ant-595

[slow shoal]

I poked around at it and it seems like the connector had a traffic change on 09/19 and another traffic change ~09/16 - but not seeing evidence of not syncing records. That is pretty unusual. It's running and saying it completed sync, but isn't getting new records?

May be a good opportunity to give fivetran a try.

I'd also suggest recreating the airbyte connector.
It seems like the airbyte connector is making requests, but at a lower frequency to before.

Is it possible you have more than one airbyte connector set up, or changed the frequency somehow?

[kind basin]

@vital nova we have a lot more faith in the fivetran connector in the long run. might be worth giving it a shot. we've been developing it with the fivetran for weeks now. what's the destination? bigquery?

[vital nova]

I did, I messaged you about it 🙂

[vital nova]

Snowflake

[vital nova]

I did just recreate the Airbyte connector (prior to starting this thread), but I'll do some more investigating to see if something funky is going on

[vital nova]

But yeah, that's the behavior I'm seeing

[vital nova]

I should also note that I’m open to suggestions if you have any regarding the destination, I’m not at all wed to Snowflake!

Airbyte connector not syncing records created after 9/10

[vital nova]

So I was running the OSS version of Airbyte on an EC2 instance—I just shut that instance down and set up a new connection using Airbyte's cloud service, and am observing the same behavior 😦

Separately, I'm also noticing that syncing fails if I try to replicate empty tables/streams. That's not a blocker for me though, simply not syncing said streams solves that problem.

Or at least prevents the syncing from failing with an error.

[slow shoal]

Cool - thanks for the info. I'm going to try and repro and figure out what's going on

[slow shoal]

I have a fix pending for the Airbyte empty tables issue! That should be fixed in the next day or two.

I'm now looking into not syncing issue in the title

[slow shoal]

The airbyte empty tables issue has been fixed! You should now be able to sync empty tables.

I have not been able to repro the records after 9/10 issue to airbyte. With permission from @vital nova - I set up a convex connector to import from their instance and was able to sync documents after 9/10

[vital nova]

Final update on this—we did sort of identify what the issue was, but it was not clear whether it was a Convex or Snowflake issue, and seemed too hard to pin down to be worth investigating further. So I've decided to just make the full-time switch now over to Fivetran! And that's been a great experience so far.

[kind basin]

great! @vital nova keep us apprised on how that continues to work out. we're putting a lot of eggs into the fivetran basket moving forward for customers who want a streaming export, so knowing that's a high quality experience is really important to us