Any Web Crawling / Scraping, Database Experts Out There?

[MustacheMusings]

I am a co-founder of a silicon valley startup looking to find a coder with strong web crawling / scraping and database expertise. We are an early stage, stealth mode startup working on something unique. I can't say much more at the moment since I do not want to reveal everything at large, but would be very interesting in speaking with a technical expert. Please contact me directly via email. Thanks!

[nanu]

Sure, this isn't shady at all. Someone signing up for a forum simply to provide an amazing opportunity just for members of this forum (and this forum has nothing to do with the expertise he's after, though admittedly, there are a lot of software engineers here).
Oh, and did I mention that they always say they are a "stealth startup working on something special/unique/disruptive that's going to be the next Google/Facebook/insert-successful-tech-company-here"?

Obviously I could be wrong. And it's also possible that Nigerian prince really does need your help.

[MustacheMusings]

I am new to this forum (and new to forums in general) and not sure if I doing this correctly...

Actually I am looking for a guy who posted in this forum some time ago, I can't seem to find him, but he discussed a server cluster he built and programmed to crawl sites - very impressive.

Sorry that you think it sounds shady. Feel free to email me and I can share my credentials. I have been in tech over 20 years and it is likey that if you a recently purchased an electronic device they may be a little of me in there.

No I am not promising this is the next facebook but I do have some investors and partners that are very interested.

[m_m]

Diffbot?

[nanu]

As I said, I could be wrong. But you have to admit yourself that such an opportunity sounds a bit too good to be true.

[MustacheMusings]

Man you guys are really beating me up here!

Admittedly I may be a little older and not up to date on forum etiquette, but come on, give a brotha a chance!

[MustacheMusings]

As I said, I could be wrong. But you have to admit yourself that such an opportunity sounds a bit too good to be true.

Ha! I haven't promised anything yet. How do you know what the opportunity is?

Do you have the expertise I am looking for? If so please contact me via email and we can discuss in more detail. Thanks!

[MustacheMusings]

Diffbot?

Nope. This is real.

[ronrico77]

check out 80legs.com

[MustacheMusings]

check out 80legs.com

Hmm, looks interesting. Can that generate results in real time based on a search query?

[Beaker]

Why are you looking for coders on a financial independence forum? Most people would think to look at, I don't know, a tech recruiting site, or an SEO site, or a software conference, or a local software-related Meetup group, or maybe something like StackOverflow Careers.

TBH, even if I was otherwise interested (which I'm not), the fact that you're going about this in such a bizarre and unlikely way would make me hesitant to work with you.

[nanu]

I just chatted a bit with MustacheMusings and there's a chance he might be legit.

Obviously, be careful who you get into bed with (literally and figuratively) and who you trust.

[MustacheMusings]

Why are you looking for coders on a financial independence forum? Most people would think to look at, I don't know, a tech recruiting site, or an SEO site, or a software conference, or a local software-related Meetup group, or maybe something like StackOverflow Careers.

TBH, even if I was otherwise interested (which I'm not), the fact that you're going about this in such a bizarre and unlikely way would make me hesitant to work with you.

Great question. I live in the Bay area and things are really hot out here. I do attend the Meetups, the pitch competitions, the incubators - it is a lot of work and most folks are tied up with other opportunities.

I was on this site checking out some financial posts and came accross a guy who described an example of a server farm he built and how he used it to collect Tbytes of data - I was actually trying to find him specifically. But I tried multiple searches and can't seem to locate him. Was hoping he (or another with similar skills) would respond.

FYI - I am already working with a team overseas but the quality is lacking and I have to hold their hand every step of the way. I need someone who is a technical guru to be my platform architect and get things done while I focus on other aspects of the business.

If you do know someone who has the skills I am looking for who also lives in the Bay area could you please send them my way? Thanks.

[MustacheMusings]

I just chatted a bit with MustacheMusings and there's a chance he might be legit.

Obviously, be careful who you get into bed with (literally and figuratively) and who you trust.

Thanks nanu! Good talking with you offline and wishing you the best for your upcoming changes!

[arebelspy]

Why are you looking for coders on a financial independence forum? Most people would think to look at, I don't know, a tech recruiting site, or an SEO site, or a software conference, or a local software-related Meetup group, or maybe something like StackOverflow Careers.

TBH, even if I was otherwise interested (which I'm not), the fact that you're going about this in such a bizarre and unlikely way would make me hesitant to work with you.

Great question. I live in the Bay area and things are really hot out here. I do attend the Meetups, the pitch competitions, the incubators - it is a lot of work and most folks are tied up with other opportunities.

I was on this site checking out some financial posts and came accross a guy who described an example of a server farm he built and how he used it to collect Tbytes of data - I was actually trying to find him specifically. But I tried multiple searches and can't seem to locate him. Was hoping he (or another with similar skills) would respond.

FYI - I am already working with a team overseas but the quality is lacking and I have to hold their hand every step of the way. I need someone who is a technical guru to be my platform architect and get things done while I focus on other aspects of the business.

If you do know someone who has the skills I am looking for who also lives in the Bay area could you please send them my way? Thanks.

If it was that recent, just search your browser history for the first threads you read here.

[MustacheMusings]

Why are you looking for coders on a financial independence forum? Most people would think to look at, I don't know, a tech recruiting site, or an SEO site, or a software conference, or a local software-related Meetup group, or maybe something like StackOverflow Careers.

TBH, even if I was otherwise interested (which I'm not), the fact that you're going about this in such a bizarre and unlikely way would make me hesitant to work with you.

Great question. I live in the Bay area and things are really hot out here. I do attend the Meetups, the pitch competitions, the incubators - it is a lot of work and most folks are tied up with other opportunities.

I was on this site checking out some financial posts and came accross a guy who described an example of a server farm he built and how he used it to collect Tbytes of data - I was actually trying to find him specifically. But I tried multiple searches and can't seem to locate him. Was hoping he (or another with similar skills) would respond.

FYI - I am already working with a team overseas but the quality is lacking and I have to hold their hand every step of the way. I need someone who is a technical guru to be my platform architect and get things done while I focus on other aspects of the business.

If you do know someone who has the skills I am looking for who also lives in the Bay area could you please send them my way? Thanks.

If it was that recent, just search your browser history for the first threads you read here.

Hi, I tried that, still could not find it. I remember it was a long thread - several pages. And I think the guy had tourette syndrome (not certain, something like that). Anyway I tried various search terms like "server", "scraping", and their combos, still no luck. Thanks fo the suggestion.

[forummm]

Why are you looking for coders on a financial independence forum? Most people would think to look at, I don't know, a tech recruiting site, or an SEO site, or a software conference, or a local software-related Meetup group, or maybe something like StackOverflow Careers.

TBH, even if I was otherwise interested (which I'm not), the fact that you're going about this in such a bizarre and unlikely way would make me hesitant to work with you.

Great question. I live in the Bay area and things are really hot out here. I do attend the Meetups, the pitch competitions, the incubators - it is a lot of work and most folks are tied up with other opportunities.

I was on this site checking out some financial posts and came accross a guy who described an example of a server farm he built and how he used it to collect Tbytes of data - I was actually trying to find him specifically. But I tried multiple searches and can't seem to locate him. Was hoping he (or another with similar skills) would respond.

FYI - I am already working with a team overseas but the quality is lacking and I have to hold their hand every step of the way. I need someone who is a technical guru to be my platform architect and get things done while I focus on other aspects of the business.

If you do know someone who has the skills I am looking for who also lives in the Bay area could you please send them my way? Thanks.

If it was that recent, just search your browser history for the first threads you read here.

Hi, I tried that, still could not find it. I remember it was a long thread - several pages. And I think the guy had tourette syndrome (not certain, something like that). Anyway I tried various search terms like "server", "scraping", and their combos, still no luck. Thanks fo the suggestion.

Did you search for Tourettes?

[gimp]

Oh my god, this is hilarious now that I see the context of your post and needs.

I understand now. I wish you the best of luck in finding your person. Plenty of search engine folks in the bay area, you will surely find some if the cash is there and the job makes sense.

[ronrico77]

check out 80legs.com

Hmm, looks interesting. Can that generate results in real time based on a search query?

I don't think anything will crawl and index in realtime - typically the indexing is done, then a search is run against the results of the index. How you do the search against the index is probably up to you.  You might look at apache lucene, and if 80legs isn't your thing - apache nutch. for open source search and open source web crawling.