Great question — answering as the UphillTech team:
Writing a robots.txt is an easy process. Follow these simple steps: Open Notepad, Microsoft Word or any text editor and save the file as 'robots,' all lowercase, making sure to choose .txt as the file type extension (in Word, choose 'Plain Text' ). Next, add the following two lines of text to your file: User-agent: * Disallow: 'User-agent' is another word for robots or search engine spiders. The asterisk (*) denotes that this line applies to all of the spiders. Here, there is no file or folder listed in the Disallow line, implying that every directory on your site may be accessed. This is a basic robots text file. Blocking the search engine spiders from your whole site is also one of the...
Full answer by jamal: https://wordpress.stackexchange.com/a/176411 (CC BY-SA 3.0).
Need it done for your business? UphillTech.
