Best approach to collecting log files from remote computers?

Question

I have more than 500 machines distributed over a global network with three continents. Periodically, I need to collect text files that are located on the local hard drive on each client. Each server runs a Windows 2003 server, and the files are mounted on a shared resource that can be accessed remotely as \ server \ Logs. Each machine contains many files, each of which can have several MB, and the size can be reduced by means of a fastener.

So far, I have been trying to use Powershell scripts and a simple Java application to copy. Both approaches take several days to collect 500 GB or so of files. Is there a better solution that will be faster and more efficient?

Answer 1

I assume that it depends on what you do with them ... if you are going to parse data for metrics data in a database, it would be faster if this parsing utility was installed on each of these machines for analysis and upload to your central database at the same time.

Even if all you do is compress and copy to a central location, configure these commands in the .cmd file and pay for the launch on each server automatically. Then you distribute the work among all these servers, and do not force your local system to do all the work. :-)

Answer 2

, , - , . , , , .

: , , , , . , / .

, . , . , , , .

Answer 3

:

• ( )
• ( ), ,
• (compress + FTP + uncompress , FTP)
• Push (FTP SMB, FTP Windows "-s: scriptfile" )
• , - .
• ( )
  • , IP- , ?

, :

• ( ?)
• / .
• ,

Answer 4

. , , , NAT . , , .

Answer 5

, , . : , ?

Answer 6

:
, :


.

, srver, :
, / .
( .)
, , , .
.

, .
/, tbh .

, , .

Answer 7

NetBIOS , , , FTP. , FTP- . , - FTP , . :

Configure the FTP server as the central collection point. Schedule tasks on each server to attach log files and FTP archives to a central FTP server. You can write a program that automates task scheduling remotely using a tool such as schtasks.exe:

KB 814596: How to use schtasks.exe to schedule tasks in Windows Server 2003

You probably want to stagger the download back to the FTP server.