Suggested Applications for SFTP/FTP uploads?

I'm working with some vendors who have very large product catalogs and while many times they have an API endpoint available to tap into, there are often limitations into the actual number of products I can pull, defeating the purpose of the API.

When I speak to these vendors they almost always point me to their XML datafeeds via SFTP/FTP. These are zipped up .txt files that sit on on a server I have access to and they update that file multiple times a day.

I know there is not a native way to have Xano connect directly to an FTP server and download the data into a Xano database table, but is there a suggested path on how to accomplish this in an automated fashion?

I know very little about the relationships between an API and FTP so maybe this isn't even a thing, but what I'm picturing is something like:
1.  Use something like https://www.ftp-api.com/ to let Xano's API connect directly to the FTP server.
2. Xano triggers a download of the file destination (almost always a compressed .gz file) and then unzips it.
3. Xano then grabs the unzipped .txt file and uploads it directly to a secondary database table.
4. Xano then runs through a function stack that scans the table with the new information for changes and uses that data to update the primary database table.
5. When it's all done, Xano clears that secondary table.
6. Xano repeats this process via background task at whatever interval I set.


I know half of this is out of Xano's services, I'm mainly wondering if this is a common thing other users have done and if so, what tools they used to accomplish it.
Other
19 replies