Offsite GB-PVR Wiki Test

[steeb]

Dear all,

one for the Web Gurus amongst you.

Say perhaps I ran pmwiki with full and current GBPVR wiki pages on a test server @ home linked to an upper domain level i.e. http://www.mydomain.com but GBPVR test pages were buried in a further folder i.e. http://www.mydomain.com/GVBPVR/Wiki/Test
are search engines still going to find it?

Will a simple robot off in html page file work? I wouldn't want this to be recognised by search engines as would just be a temporary thing.

I know some stuff but not enough

Cheers

Steeb

[whurlston]

Just use a robots.txt in the top level folder:

http://www.mydomain.com/robots.txt

Code:

User-agent: *
Disallow: /GVBPVR/Wiki/Test

[steeb]

Excellent Whurlston,

'twis what I thought something to do with 'robots off' but confirmation of this is great!

But you are far more clued up than me :o

Just to be sure!!!

link would be something like this:

http://www.mydomain.com/GBPVR/Wiki/Test/

Code from:

http://www.mydomain.com/robots.txt

goes in head of page in upper level folder?

Job is a good 'un? (sorry UK talk!)

Cheers

Steeb

[steeb]

Ah ha I see what I've done now, I didn't realise http://www.mydomain.com was a real site! I just used it as example.

Let's say http://www.oldsteeb.co.uk on same premise!

cheers

Steeb

[whurlston]

Yup, I was just using the domains that you used. robots.txt would be in the root of whatever domain you use for /GBPVR/Wiki/Test

You would place the the text from my code block in your robots.txt, don't use the one from mydomain.com (I didn't test it to see if it was a real domain either.)

[pz1]

Just use a robots.txt in the top level folder:

http://www.mydomain.com/robots.txt

Code:

User-agent: *
Disallow: /GVBPVR/Wiki/Test

Be careful, not all search engines will honour that request.

[whurlston]

Be careful, not all search engines will honour that request.

Correct, there are some bad search engines that will ignore it but the major ones honor it. And since his test site will be up only temporarily, I didn't see it as enough of an issue to get in depth.

[johnsonx42]

Correct me if I'm wrong though, as long as the top-level site has no links to the lower-level site, and Directory Browsing isn't enabled, then any lower-level directories can't be crawled. There's no way for the crawler to know about the directory.

[martint123]

Correct me if I'm wrong though, as long as the top-level site has no links to the lower-level site, and Directory Browsing isn't enabled, then any lower-level directories can't be crawled. There's no way for the crawler to know about the directory.

But all it needs to do is find a link on somewhere like here and it has then off it goes.

[johnsonx42]

But all it needs to do is find a link on somewhere like here and it has then off it goes.

True, I should have said 'as long as no site has any links'.