Author Topic: No coding experience, but need a simple code/script thing - help?  (Read 3365 times)

Landor n Stella

  • Stubble
  • **
  • Posts: 103
  • Location: Utah
    • Heartland House Project Blog
Totally unrelated to MMM stuff... I have a program that I use to scorekeep for an event that generates .dat files. On an hourly basis (or so), I need to quickly merge the .dat files so that I can have three separate versions of the program going, but that there is a single outcome after the merge. I don't know anything about coding, but it seems to be a simple thing. The .dat files are tab-separated text columns that store the data, and each row is related to a single object that is tracked linearly. Can anyone point me to a resource that would help me code something myself, or where I could hire somebody to do the job for me?  Feel free to message me for more details or information.
Thanks!

spider1204

  • Stubble
  • **
  • Posts: 244
  • Location: Lexington, KY
Re: No coding experience, but need a simple code/script thing - help?
« Reply #1 on: November 12, 2012, 01:24:53 PM »
If programming was something that you always wanted to learn you could probably start learning it and one of the earlier lessons will probably about reading and writing to and from files.  You would then have to learn how to program the logic to perform the merge, but I don't think there will be a resource I could point you to that would directly tell you this.  There a lot of great getting started with programming resources out there, I can dig some up later for you.  It's probably something you'd only want to do if you already have an interest in learning this skill though.

Also, there was a post a while ago about a programmer looking to make extra spare money in his spare time, maybe you could work out a deal with him?  I might be interested myself as well.
« Last Edit: November 12, 2012, 01:26:52 PM by spider1204 »

TLV

  • Bristles
  • ***
  • Posts: 492
  • Age: 36
  • Location: Bellevue, WA
Re: No coding experience, but need a simple code/script thing - help?
« Reply #2 on: November 12, 2012, 01:34:57 PM »
Are there column headers (or anything else except data in tab-delimited rows)? What exactly needs to be merged?

Would opening each of the files in a text editor (e.g. Notepad) and copying/pasting the entire contents of each into one file solve the problem? (I'm not asking if that would be efficient enough, just if it would produce correct results.)
If so, on a unix/mac system (or windows if you install MinGW) you could run the following on the command line:
cat file1.dat file2.dat file3.dat > merged.dat

 
« Last Edit: November 12, 2012, 01:45:13 PM by TLV »

spider1204

  • Stubble
  • **
  • Posts: 244
  • Location: Lexington, KY
Re: No coding experience, but need a simple code/script thing - help?
« Reply #3 on: November 12, 2012, 01:41:54 PM »
http://www.learnpython.org/

TLV's solution sounds pretty awesome if it produces correct results.  You could pretty quickly create a script that will just run that command every hour.

gdborton

  • Bristles
  • ***
  • Posts: 278
  • Age: 35
  • Location: Los Angeles, CA
Re: No coding experience, but need a simple code/script thing - help?
« Reply #4 on: November 12, 2012, 04:20:10 PM »
If I'm reading this correctly, you want to run three applications that each generate a .dat file, a script to combine them (the part you need built), and then have another program to parse the merged file.

What OS are you on?  Do you need to continuously update the merged file with new data, or can it be merged then parsed and forgotten?

Landor n Stella

  • Stubble
  • **
  • Posts: 103
  • Location: Utah
    • Heartland House Project Blog
Re: No coding experience, but need a simple code/script thing - help?
« Reply #5 on: November 13, 2012, 06:58:58 AM »
Thanks for the quick response! I'm not really interested in learning coding, so I will proceed with the idea that someone could do this for me. And here's the answer to your questions:

Quote
Are there column headers (or anything else except data in tab-delimited rows)?

No, there are not column headers. :-( I don't even know what each of the columns are for, really, I just know some of the columns because occasionally I open the .dat files and edit the text directly. And I've figured it out over time. There is, however, always a unique identifier for each row.

Quote
What exactly needs to be merged?

The program I use is a software that keeps track of people in a tournament. It identifies each person with a unique 10 digit number. As they progress through the tournament, it stores information about how they did (win/loss/draw) and it also tracks how their opponent did each round. So the files are continuously updated throughout the day. The problem I am trying to solve is having just one person running the software. I would like to have up to 3 people running the software at a time during a round, and then at the end merge all of the .dat files from those three sources into a single master file, start the next round, and then open three copies again.

Quote
Would opening each of the files in a text editor (e.g. Notepad) and copying/pasting the entire contents of each into one file solve the problem? (I'm not asking if that would be efficient enough, just if it would produce correct results.)

I don't think it would produce the correct results. For example, if you and I are running the software together and there are 5 people in our tournament, my files would have the information for competitor #1, 3, and 5, and your files would have the information for #2 and 4.  Each of us would have a full copy of the master file, so in my file there would be placeholder 0's in the #2 and 4 rows, and yours would have the placeholders in 1, 3, 5. Copying and pasting (I think) would mean that we would have 10 rows of information, and they would be out of order since each row is related to a specific person. I think what needs to happen is like an overlay - one set of files would be checked against the other, and lines with no data would be replaced with data from the other if it lined up. So in our mock scenario, the end of the merge would result in 5 rows of data in these tab columns, with rows 1, 3, and 5 coming from my file and rows 2 and 4 coming from yours. There is more than just one .dat file that needs to be merged like this each hour, too. I would have to look to be sure, but probably about 4 or 5 files total.

Quote
What OS are you on?  Do you need to continuously update the merged file with new data, or can it be merged then parsed and forgotten?

I run windows 7. The program I have was built for a 32 bit system but I have a patch that forces 32 bit on my 64 bit. Yes, the files are continuously updated with new data. At the end of the day, the files will hold all of the information about the competitors and the tournament, so the files are getting added to/modified all day. 

keith

  • Bristles
  • ***
  • Posts: 311
  • Age: 39
  • Location: Seattle/eastside
Re: No coding experience, but need a simple code/script thing - help?
« Reply #6 on: November 13, 2012, 05:19:46 PM »
Ok so you definitely need a real merge, and not a copy/paste file combine.

I can volunteer to write a merge script in PowerShell for you, since Windows 7 includes this natively without having to install anything (64-bit vs 32-bit would both be supported).

What I will need to proceed is the following:

1. Two or three different sample output .dat files that would need to be merged. Ensure that the sample files have at least 2 rounds already played (so I can see how results from different rounds are applied to the same file). You can PM me or attach to the thread perhaps?

2. Output of a manually merged file (to see the final result of what it should look like, including ordering, etc)

Once I have that, I can whip up a script.