It can take some time

Took me a while

it's rather straightforward, but i still took a couple minutes more than necessary

What does disallow not tell a robot

Nice Challenge!

[REDACTED] This comment is only shown to users who have solved this challenge.

[REDACTED] This comment is only shown to users who have solved this challenge.

nice

It's really simple if you actually read the question. Took literally a minute or two.

The wording is really what throws me off with some of these questions.

hard one : |

robots can find webpages?????

[REDACTED] This comment is only shown to users who have solved this challenge.

[REDACTED] This comment is only shown to users who have solved this challenge.

what file do web crawlers view to index the site

I learned a lot about how webcrawlers work in the course of doing this challenge. It was very much worth it to me. On one of the sites it said "of course if you are a hacker you will want to see which pages are disallowed" which makes sense to me.

[REDACTED] This comment is only shown to users who have solved this challenge.

can any one tell me how to improve my ctf skills

My favourite so far

it was a good one

just think about robots

really man thnx this is really helping me a lot i just research things deeply and boom get some deep knowldege

the answer is a filename or what?

It took me a while to find the hint. The "robot" and the "disallow"

Nice.

Interesting.

Mr robot ;)

gave me a headache

super easy

Ez

Im noobie in CTF etc and that was hard for me, but i learned what robots.txt do. So nice, if anyone have problem good luck :)

I’m a noob and the research for this taught me a great deal. Took me a long time to think outside the box and finally get it. Loved it.

for putting in the flag is it just the answer or is it flag{answer} or something else

kinda hard but ok

Definitely took a while but enjoyed it.

Damn.

Good!

[REDACTED] This comment is only shown to users who have solved this challenge.

Super easy

not easy :)

its better toprovide solutions after end and better to keep a option like give up and show answer

easy

nice

[REDACTED] This comment is only shown to users who have solved this challenge.

ok

That was nice. Enjoyed it. Learn about google robots.txt and stuff if you are stucked.

I may be dumb, I don't know, but I really am struggling with this. I am new to hacking in general, yet I understand this really isn't that hard. I've been struggling for like an hour so can anyone please help me.

First time I heard about robots, I am noobie and it can be hard, but I loved it!

Nice challenge for learning about web crawlers

thanks for hints in comments

Hard one

it took 4 seconds to solve it

nice one!

Really good task for all the newbies. It took me much time and I rad all the comments below and it helped me a lot.

Man, this challenge was <strong>SUPER COOL</strong>, fantastic... thanks for the comments an thanks to google shared knowledge, i learned a lot.

<strong>Good</strong>

Good

<h40>Good</h40>

<h4>Good</h4>

took some brain cycles

Awesome

I actually got it. I'm getting better at this. 😂😂

Very easy one, answer to the question is the place for clue.

This is a good one!

a

Hint for people who are stuck: You need to find a file

Nice! Just need to think!

I'm an beginner. Great challange :)!

Don't overthink it. It's straightforward

hi, i have a question why flag incorrect even tho i'm in robots.txt.. i'm still confused.. sorry i'm new

<strong>Skynet</strong> is ~~active~~

Gotta say, this is more of a puzzle than some of the others but it's creative enough that I have to give it credit.

this was fun for a beginner like me

this was fun for a beginner like me

AAAAAA

<img src="https://i.kym-cdn.com/entries/icons/facebook/000/021/807/ig9OoyenpxqdCQyABmOQBZDI0duHk2QZZmWg2Hxd4ro.jpg" onmouseover="console.log(123)"></img>

I cant get this flag to work, is there a certain format i need to use?

noob challenge

[REDACTED] This comment is only shown to users who have solved this challenge.

<img src="" onerror=alert()>

Not so intuitive but a great question to think out of the box.

YO DUDE

Nice one! The wording threw me off guard really

Nice one! The wording threw me off guard really

That was funny

Never would have guessed

Silly but nice challenge had me confused a bit

really easy but i now understand why i needed to "think out of the box", still i learnt how robots work and enjoyed this

[REDACTED] This comment is only shown to users who have solved this challenge.

Keep going if you think you are on the right track

[REDACTED] This comment is only shown to users who have solved this challenge.

It wasn't this hard because i know alot about web things if you cannot find it search "how to keep web crawlers out"

am complete newbie idk how i solve it

A solid one (robots.txt)

need a bit

Very easy if the first line "Where do robots find what pages are on a website?" clicks for you. If not, you have to do some research on exclusion standards

is the answer not robots.txt cause im sure it it but its not working for me

Excellent! It can get experiences for me.

Another him go to ctflearn.com/robots take a look it the html turn it into another url

Another him go to ctflearn.com/robots take a look it the html turn it into another url

Many thanks, i've learned lot from this.

[REDACTED] This comment is only shown to users who have solved this challenge.

I didn't get it at first but it's easy as hell when you get to the first step

I might have added "on this website" to the question before the hint

nice one

nice one

[REDACTED] This comment is only shown to users who have solved this challenge.

using robots.txt

Ezz

Self explanatory actually.

gobuster is the key to go

Its very easy

co0ol

took me a whileto figure out what to do

easy

Very interesting!

ctflearn.com/robots.txt

Good one. You need to know a bit of indexing behavior but nothing complex.