Ultra Lightweight open source FTP client

[BrownOut]

Does anyone know of a very, very lightweight open source FTP client available? All I want to do is stream data from a file to my embedded project. Ideally, a "bare-bones" client written in C would fit the bill.

 

[BrownOut]

Before anyone answers, I found this:

If I'm not mistaken, the FTP server is usually implemented as a client of the inetd/xinetd daemon. As such, it should not do any socket-level IO, only reading and writing to standard input & standard output.

So I am probably asking for the wrong thing. I just want to stream some file data to my embedded design. Guess I'm back to 'rolling my own.'

 

[BrownOut]

Ok, I got that done. NOW, can anyone recommend software that will measure throughput for my 'home made' ftp program. I want to stream document size text files and measure performance. Thanks!

 

[Robin2]

What about a simple program that records the clock time, sends or receives some data, records the time again and calculates the throughput rate based on the elapsed time and the number of bytes sent or received?

 

[BrownOut]

Hi Robin2, thanks for your reply. I can add something like that to my program. It wouldn't be that hard to do. I wonder how accurate it would be since some of the time has to be devoted to processing the time and byte count, etc. I was sort of hoping for an independently executing 'bus sniffer' Would 'wireshark' have this capability? Has anyone used wireshark?

 

[Robin2]

I do a bit of hobby programming in Ruby which is an interpreted language and therefore relatively slow. As far as I can see it gets the time so quickly that the overhead is irrelevant unless you are trying to measure very short periods (fractions of a millisecond). You can minimize the "cost" of the calculation by downloading a larger block of data - that way the fixed calculation error is spread over more bytes. I was using this method to see how quickly I could send about 4k bytes of data over a USB connection to a FTDI UM245R device.

 

[Sceadwian]

What is the software running on? All modern OS's should have built in or free downloads for bandwidth monitoring.

 

[BrownOut]

It's running on Windows XP.

 

[BrownOut]

I do a bit of hobby programming in Ruby which is an interpreted language and therefore relatively slow. As far as I can see it gets the time so quickly that the overhead is irrelevant unless you are trying to measure very short periods (fractions of a millisecond). You can minimize the "cost" of the calculation by downloading a larger block of data - that way the fixed calculation error is spread over more bytes. I was using this method to see how quickly I could send about 4k bytes of data over a USB connection to a FTDI UM245R device.

I'll give that a try. However, I'm still interested in knowing about any peroformance monitoring software I can use. Thanks!

 

[Sceadwian]

XP has a built in resource monitor that can be accessed by opening the task manager and selecting the network tab.

 

[Sceadwian]

Duplicate post, sorry

 

[BrownOut]

I'm trying to record the data rate for comparason between runs. Here is the output form my program, recording the raw numbers as we've discussed. Still not sure how accurate this is, and why the data rate is so low. Not sure if the problem is my embedded system or my traffic generator ( which is my laptop running a version of telnet )

Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.281000 seconds
5.99 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.282000 seconds
5.97 Meg bits per seconds
Data Sent: 210450 bytes in 0.282000 seconds
5.97 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.281000 seconds
5.99 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.281000 seconds
5.99 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.281000 seconds
5.99 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds
Data Sent: 210450 bytes in 0.281000 seconds
5.99 Meg bits per seconds
Data Sent: 210450 bytes in 0.297000 seconds
5.67 Meg bits per seconds

 

[Robin2]

Can you try sending different quantities of data - half, quarter, fifth to see if the timing is consistent, or at least that the inconsistencies are understandable. This should help to create confidence in the rate measurement before you spend time looking for problems elsewhere.

 

[BrownOut]

I can do that.

 

[BrownOut]

Hi again. Well I tried an experiment to see what the data rate would be for a simple client-server running on my laptop. It is just an attempt at a 'sanity check' just to see if the numbers I'm getting are reasonable or not. The best I could do was ~10MBPS. Hmmm...? I thought it would be more. Here is the code I used in the client program to send the data and calculate the rate. Would anyone be so kind as to check the calculations? I've added comments to help figure out what the code is doing. It's easy, it reads a text file in the first 12 lines or so, and appends some terminating characters for each line. This it records the local time and sends the file 30 times. After that, it records the local time again and calculates the elapsed time for sending the data. The results is stored in the variable "et". Then the printf/fprintf statements calculate and report the data rate. That's where the math takes place. Thanks!

 //Get the text to send from a file
		do{
		   fgets(finpData, 1023, data_file);
		   if(!feof(data_file)) {
		     printf("%s", finpData);
		     numbytes += strlen(finpData);
		     fDatalen=strlen(finpData);
		     finpData[fDatalen-1]=0xd;
		     finpData[fDatalen]=0xa;
		     finpData[fDatalen+1]='\0';
		     strcpy(dtextbuf[prowdtext++], finpData);
		   }
		}while(!feof(data_file));

		prowdtextmx=prowdtext;
		printf("Document read %d lines\n", prowdtextmx);

//We will continue to run the test as long as "go_again" is == 1.  We enter this at the bottom
//of the loop
		while(go_again == 1) {
//Get starting time
		  GetLocalTime(&lt);

//the vairable "timestosend" is set to 30
		  for(i = 0; i < timestosend; i++){
		   for(prowdtext=0; prowdtext < prowdtextmx; prowdtext++) {
		     send(sock, dtextbuf[prowdtext], strlen(dtextbuf[prowdtext]), 0);
		   }
		}

//Get ending time
		  GetLocalTime(&lt1);
//Calculate time elapsed
		  et=(((int)lt1.wSecond + .001*(int)lt1.wMilliseconds) -((int)lt.wSecond + .001*(int)lt.wMilliseconds));
//Print the results & save to results file
		  printf("Data Sent: %d bytes in %f seconds\n", i*numbytes, et);
		  printf("%2.2f Meg bits per seconds\n", 8*(i*numbytes/et)/1000000);
		  fprintf(results_file, "Data Sent: %d bytes in %f seconds\n", i*numbytes, et);
		  fprintf(results_file, "%2.2f Meg bits per seconds\n", 8*(i*numbytes/et)/1000000);

		  printf("Go Again?\n");
		scanf("%d", &go_again);
		}
		fclose(data_file);
		fclose(results_file);

 

[Robin2]

I presume you are using C and I am not expert at that so I hope these comments make sense.

Is there a time penalty for sending a single byte at a time? I suspect there is an overhead associated with a "send" which would be the same whether you send one byte or 1000. If USB is involved there could be a huge penalty for short strings.

Why are you including strlen(...) within 'send'. It should always be 1, so why not avoid the cost of the calculation. And if you do need to calculate it try to do so outside the loop.

If you must send single bytes it might be worth experimenting with sending (say) just the letter 'A' to avoid the 'dtextbuf[prowdtext]' in case that has a time cost (though I suspect it is tiny).

Another useful test might be to run all of the program except the 'send' line to see how much time the other parts take (again, probably not much, but worth checking)

 

[BrownOut]

Thanks Robin2,
I'm not sending a single byte at a time. I read lines of text from a file and send each line, one at a time. It's possible that I could send multiple lines and get a little speedup. At this moment, I'm only concerned that my code accurately calculates the data throughput. In the actual application, I might buffer a block of data and send it all at once, but of course, that increases latency, so I need to decide if that would be a problem. So right now, I'm just concerned about measuring the performance correctly.

If you're convinced I'm sending only a byte, then I'll take another look at the code. But I'm sure I'm sending lines of 20-30 bytes or whatever is typical of text documents.

Another useful test might be to run all of the program except the 'send' line to see how much time the other parts take (again, probably not much, but worth checking)

That's not a bad idea. I assumed the send was the bottleneck, but it wouldn't hurt to verify.

 

[BrownOut]

Hi again, I found an interesting document that describes how to accelerate system ethernet throuput HERE. Here is a summary, for informational purposes:

DMA engine for moving data to and from the Ethernet device
■ Increasing the overall system frequency, including components such as the
processor, DMAs, and memory
■ Using low-latency memory for Nios II software execution
■ Using a custom hardware peripheral to accelerate the network checksum
■ Using fast packet memory to store Ethernet data

 

[Robin2]

Thanks Robin2,
If you're convinced I'm sending only a byte, then I'll take another look at the code. But I'm sure I'm sending lines of 20-30 bytes or whatever is typical of text documents.

I think "dtextbuf[prowdtext]" represents a single byte at position "prowdtest". But I am not good at C.

 

[BrownOut]

Hi Robin2,

I understand why you think that. I didn't show what the variable really is, my bad. It's a 2-dimentional array, it's declration is this:

char dtextbuf[1024][256];

So 1024 lines and 256 characters per line. When I read the text file, I use: fgets(finpData, 1023, data_file); The fgets statement copies characters until it reaches the 'newline' character. So, each call to dtextbuff[prowdtext] will send to up 256 characters.

BTW, I just realized I have an inconsequential error in the fget statement. The second argument should have been 255.