I'm finding that my web site is being hit by a number of unwanted crawlers (cyveillance, etc). I would like to find a simple way to deny access to these bots that I can impliment via a file in my root directory.

The .htaccess file for the Apache server sounds exactly like what I need, but I'm on a Windows server with a commercial hosting company.

Is there a file or protocol similar to the .htaccess file for windows server?
Back to top

Hi Tom, unfortunately, you're using ASP on your site. If you weren't, there would be no particular reason to use Window's hosting, and many excellent reasons to switch to Linux/Unix type hosters, running Apache. But you're kind of stuck.

Short answer, if you are on a shared server, there's nothing you can do. .Htaccess is simply one of the rewards you get for using apache servers, it gives you a level of control, especially on shared servers, that Windows can only dream about.

Personally, I never use windows hosting for any reason, and in fact just got my last site, a client's, off Windows today, what a relief.., with apache you'll save money long term, you'll have more powerful options, and you'll probably have better uptime. We're big fans of pair.com, but you may not need that high level of performance and reliability, there are others out there that are fine too, mediatemple for example.

PS, you might think of getting some form input data validation going on your aspx page, it's easy to crash.
Back to top

Thanks for the quick reply Jeff.

I was affraid that I was going to find that the file was an Apache exclusive thing.

As for page validation, I appreciate the input. What page did you have trouble with? The only form is on the "Orders" page and I though it was pretty well vaidated.

I was messing around with the site and security a few minutes ago and did lock my self out for a minute (oops). Hopefully, you were hitting the site when I was messing it up.........

If there was a specific page and sequence you did and crashed, please let know.

Thanks for the original information.
Back to top

No, it's not that type of validation, it's validating the data input into the script on the server side from your forms using server side input data validation, not with javascript, I figured it wasn't there so I entered some characters, ' and " in this case, that will always cause a database to crash if the input data hasn't been protected against that type of attack, into the form and it crashed the program as I thought it would.

Try it, enter 'lj" or something into your order number form field on check order page, and you'll see what I mean.

Somebody more motivated can usually use the same method to hack into your site, or at least wipe out your data. That's on your aspx pages. Just a heads up to avoid future problems you might run into if you're unlucky.

By the way, you definitely want to update your forums too, anything below 2.0.13 with phpbb can be taken over instantly by anyone who wants to, for a while they were running automated attacks against phpbb installs that would wipe out the databases and just post a 'you've been hacked' type message in red and black.

In the php / mysql world that's known as mysql injection, which involves putting in a ' to close the data, then adding a line of command that will help crack the server after it, I don't pay much attention to asp stuff so I'm not up on it's weaknesses, I tend to avoid all MS products when it comes to the web due to their poor server security track record, but hackers tend to pay much closer attention to it for the same reasons.

Re your original question, if you have root access to IIS, in other words if you run the server yourself, not as shared, you can get isapi modules that allow for rewrite rules to be implemented, but again, I don't pay much attention to that side of things since Apache supports this in any type site through mod_rewrite module, which can be run through either a text file called .htaccess, or through httpd.conf, which is the main apache configuration file. I like things to be easy.

Once you have an .htaccess file, all you have to is change a few things on it for every new site and it's up and running, no fuss, no bother, free to do more interesting thing.

I doubt spiders are costing you that much bandwidth though judging by the hit counter on the home page, which you might think of taking off the page, it's better to use server stats for that kind of information, hit counters have become somewhat passe nowadays, and in your case the count is pretty low and might actually end up losing you work since people might think, oh, not many people go to this site.

Re: Page validation
In terms of actual page validation, of the HTML source code, these pages will not validate [that's the w3c html validator page] in the current format, they're not close, but since the layout is pretty simple it doesn't matter that much, most browsers handle bad html pretty well.
Back to top

Jeff,

Again, thanks for taking the time to provide such a complete response.

I did go the the Status.aspx page and tried to enter 'and" into the Order Number field. Input stopped at the 4th digit (d).

I tried hitting enter with this entry and I received a pop-up saying "Order number is 4 digits".

I tried just entering 'lj" and got the same pop-up.

I'm obviously missing something in your explantion.


Thanks for the tip on the message board. I'll chase down the later version.

Counter? You are right...I really serves no purpose. I doubt that it drives away business since its such a narrowly defined site. Only antique clock geeks would be there in the first place and there actually aren't many of those........

I really appreciate the time you have taken to review my question and site. I will continue to work on it between clock projects.
Back to top

The popup is created by javascript client side validation [happens on your browser that is, not on the server]. Turn off your javascript, enter some bad data, and you'll see the problem. Although client side validation is more user friendly, it has to be coupled with server side validation, since anyone can and will turn off javascript to do what they want to do.

Critical things server side: either strip out or escape ' and ". If the form field is only supposed to accept numeric data, test for numeric data and reject any submission from that field that is not numeric.

Think of it this way: javascript validation is to make a nice user experience, server side [asp in your case] validation is for the server, to protect if from bad data. When creating validation, the server side stuff is most important, and the client side stuff is just a nice add on feature for the user, but should never be relied on to do the job alone.

Good luck on your site.
Back to top

OK.........

Takes a while, but I get it..........
I had used aspx "required field validators" and "regular expression validatators" on the page to check the order number entry values and the email formats. But you're right, that only generates javascript. With javascript turned off, I was able to reproduce the error just like you said.

To the server side. I reviewed the code and found that I originally had a "Try/Catch as Exception" block to catch any problems incomming from client side. Unfortunately, I had them commented out for some reason......

Anyway, un-commented the Try/Catch and was not able to crash when bad data was entered. I created a "Response Write" block to send a message back to the client, but I guess it won't do much good if they have javascript turned off. Have to think about how to alert the client that they entered bad data.

Thanks for sticking with this til I understood the issue. I need to check other aspx pages to make sure I kept the Try/Catch blocks in place. I think that restoring 200 year old clocks is a lot simpler....

Thanks again..........
Back to top

LOL... glad you're getting it working, might save you some big headaches and nasty surprises in the future.
Back to top