14. Networking and Other Topics

What We Will Cover Announcements Please silence all cell phones and pagers When class is over, please shut down your computer if it is on Continuations 14.1: Improving Verification With Patterns 14.1.1: About Regular Expressions 14.1.2: Using the preg_match() Function 14.1.3: Defining Character Patterns 14.1.4: Building Regular Expressions That Work Exercise 14.1 14.1.5: Summary 14.2: Scripting the Internet 14.2.1: Sending Email 14.2.2: Verifying Network Information 14.2.3: Reading Pages from a URL 14.2.4: Parsing a Web Page 14.2.5: Uploading Files 14.2.6: Summary Exercise 14.2 14.3: Finishing the Course 14.3.1: About the Final Project Presentation 14.3.2: Lecture Finale Wrap Up Due Next: Final Project Report and Presentation (6/1/10) Continuations Questions from last class? Questions about your project? ^ top 14.1: Improving Verification With Patterns Learner Outcomes At the end of the lesson the student will be able to: Use PHP pattern-matching functions Code regular expressions to match string patterns ^ top 14.1.1: About Regular Expressions Many programming problems require matching a pattern in string variables Verifying the data received from HTML forms is one such problem For example, if you are expecting an email address, your script needs to verify the string meets requirements for email addresses john.doe@hotmail.com Regular Expression Standards There are two main standards for regular expressions: POSIX and Perl PHP supports both standards We will use Perl-compatible functions and focus on using preg_match() Commonly Used Pattern-Matching Functions (Perl Compatible) Function Description preg_match() Searches a string for matches to a regular expression. preg_replace() Searches a string for matches to a regular expression and replaces them with the specified text. preg_split() Searches a string for boundaries matched by a regular expression and splits the string into an array of strings along the boundaries. ^ top 14.1.2: Using the preg_match() Function You use preg_match() to search a string for matches to a regular expression If the regular expression pattern matches a part of the string, then it returns the number 1 (meaning true) Basic Syntax int preg_match(string pattern, string subject) pattern: regular expression pattern subject: the string to search for pattern matches returns the numbers 0 or 1 Example Using the preg_match() Function <?php $pattern = "/se/"; $subject = "She sells sea shells"; $found = preg_match($pattern, $subject); if ($found) {     echo "Matches"; } else {     echo "No match"; } ?> Try it! Put the regular expression pattern between forward slashes: / / If the pattern "se" is found, then $found is set to the number 1 Otherwise, $found is set to the number 0 Setting Pattern Modifiers By default, regular expressions are case sensitive To make a regular expression insensitive to case, use the i modifier: /pattern/i PHP supports other pattern modifiers as well: Pattern Modifiers Testing Regular Expressions As we develop regular expression, we need a way to test them We can test our regular expressions with a simple form script Try it! (See the code) For example, we can try the following pattern on our test page: /she/i Keep this form open to test the examples in the following sections More Information preg_match() ^ top 14.1.3: Defining Character Patterns So far our regular expressions only match exact strings We can do this same task by just using string function: strcmp() However, regular expression can do more than just match exact strings In this section we look at using special characters to allow us to match text in other ways These special characters are often called metacharacters Character Boundaries Character Description Example ^ Matches the beginning of a text string /^sea/ matches "sea shells" but not "shells from the sea" $ Matches the end of a text string /shells$/ matches "sea shells" but not "shells from the sea" \b Matches a word boundary /\bart/ matches "art" and "artist" but not "kart" or "start" \B Matches the absence of a word boundary /art\B/ matches "artist" but not "art" Character Types You can use regular expressions to indicate the type of a character There are three general types of characters: digits, word characters, and whitespace Character Description Example \d A digit (from 0 to 9) /\dth/ matches "5th" but not "path" \D A non-digit /\Dth/ matches "path" but not "5th" \w A word character (letter, digit or underscore) /\w\w/ matches "to" or "A1" but not "$x" or "*" \W A non-word character /\W/ matches "$" or "&" but not "A", "b" or"3" \s A whitespace character (space, tab, newline, carriage return or form feed) /\s\w\s/ matches " A " but not "A" \S A non-whitespace character /\S\S\S/ matches "123" or "abc" but not "1 3" or "a c" . Any character except newline /./ matches any single character except newline Character Classes While character types are useful, sometimes you want to limit the allowed characters to a few select letters or digits For this you can use square brackets [] to specify a character class A character class defines a set of characters that can match a single character of the text string For example, to match all the vowels in a string: /[aeiou]/i You can negate (reverse) the meaning of a character class by putting a ^ as the first symbol of the class For example, to match all the consonants in a string: /[^aeiou]/i Also, you can define a range of characters by separating the starting and ending characters with a dash Since characters and numbers are arranged in alphabetical order, you can specify all lowercase letters using: /[a-z]/ For uppercase letters you would use: /[A-Z]/ For both lowercase and uppercase letters you would use: /[a-zA-Z]/ For all letters and digits you can use: /[0-9a-zA-Z]/ Repeating Characters So far our regular expressions match a single character However, you can use metacharacters to specify multiple characters Character(s) Description Example * Repeat 0 or more times /\s*/ matches 0 or more consecutive whitespace characters + Repeat 1 or more times /\s+/ matches 1 or more consecutive whitespace characters ? Repeat 0 or 1 times /colou?r/ matches "color" or "colour" {n} Repeat exactly n times /\d{5}/ matches a five digit number {n,} Repeat n or more times /\d{5,}/ matches a number with at least five digits {,m} Repeat no more than m times /\d{,5}/ matches a number with no more than five digits {n,m} Repeat at least n but no more than m times /\d{5,9}/ matches a number with 5 to 9 digits Escape Sequences Sometimes you need to use one of the metacharacters as a literal character in a pattern For instance, you may want to search for a forward slash ('/') in a string In these cases, to search for characters that are usually treated specially, you use an escape sequence An escape sequence is a special character inside a text string that tells the regular expression parser to interpret what follows as a regular character and not a metacharacter The escape character in JavaScript is the backslash ('\') For example, to search for a date pattern like 11/13/2007 we can use: /^\d{1,2}\/\d{1,2}\/\d{4}/ The following are some of the possible escape sequences Escape Sequence Example \/ /\d\/\d/ matches "2/3" but not "23" \\ /\d\\\d/ matches "2\3" but not "23" \. /\d\.\d\d/ matches "1.23" but not "123" \* /\d\*\d/ matches "1*2" but not "12" \+ /\d\+\d/ matches "1+2" but not "12" \? /\w{5}\?/ matches "hello?" but not "hello" \n /\n/ matches a new line in the text string \t /\t/ matches a tab in the text string Alternate Patterns and Grouping Sometimes you want to define two or more patterns for the same text string For this we can use the alternation symbol ('|') The alternation symbol matches either the pattern on the left or the right For example, if we want to match either Dave or David we could use: /Dave|David/ Notice that the alternation applies to all the characters on the left or right side, and not just a single character If we want the alternation to apply to a subpattern, then we can group characters with parenthesis For example, we could modify our Dave or David example to use: /Dav(e|id)/ Another benefit of grouping is that parenthesis are remembered elsewhere in the pattern This is known as creating a back-reference You can reference these grouped subpatterns using the syntax \groupNumber Where groupNumber is the number of the grouping counted from left to right For example, if you wanted to search for repeating words in a string (a common error) in a text string like: products for for sale To create a pattern to find a single instance of a word, we use something like: /(\b\w+\b)/ Since we are looking for consecutive words separated by a space, we add a space and back-reference to the pattern: /(\b\w+\b)\s+\1/ ^ top 14.1.4: Building Regular Expressions That Work Regular expressions are very powerful -- but can be almost unreadable To build complex regular expressions, start with a simple expression After a simple start, refine your regular expression incrementally Build it one piece at a time and test each addition as you go Incremental Refinement Example This example incrementally builds a regular expression for form verification We want to verify that a form field meets requirements for email addresses The steps that follow detail a process for building this verification incrementally Determine the precise rules for your field john.doe@hotmail.com You determine what is valid and invalid input by examining email addresses and reading specifications. Some of the rules you come up with are: User names can have almost any printable ASCII character An @ symbol seperates the user name from the domain name Domain names can have letters, digits, and hyphens Each part of a domain name is separated by a dot Set up your test environment Next you build a form with an element to verify and the receiving function. You decide to use the FormVerifier class and add a verification function like that shown. Make sure these work before you add regular expressions. function isEmailAddress($field, $msg) { $value = $this->getValue($field); $pattern = "/.+/"; if(preg_match($pattern, $value)) { return true; } else { $this->addError($field, $value, $msg); return false; } } Code the most specific term possible You look at the rules and code the most specific line you can easily come up with. Then you test the regular expression to verify it works. $pattern = "/[_a-z0-9+.-]+@([a-z0-9-]+\.)+com/i"; Set anchors if you can Add the ^ and $ quantifiers where possible. This prevents characters before and after the acceptable pattern to be invalidated. $pattern = "/^[_a-z0-9+.-]+@([a-z0-9-]+\.)+com$/i"; Get more specific if you can, testing each addition carefully You may decide to restrict the top level domain (TLD) to only those authorized. This turns out to be quite complicated. Almost every two-letter combination is used by some country. In addition to the well-known generic TLD's of com, edu, net, org, mil and gov, there are many new TLD's: biz, info, name, coop, aero and museum. More are being suggested and adopted every year. We leave the coding of a TLD regular expression as an exercise for the student. ^ top Exercise 14.1 Develop a regular expression for verifying top-level-domain (TLD) names in email addresses. Make sure your regular expression works with the email pattern we have devloped so far. $pattern = "/^[_a-z0-9+.-]+@([a-z0-9-]+\.)+com$/i"; You may use this test script to test your changes. <?php main("Regular Expression Tester"); // Control the operation of a page function main($title = "") {     $msg = "";     $pattern = "/^[_a-z0-9+.-]+@([a-z0-9-]+\.)+com$/i";     $subject = "john.doe@hotmail.com";     if (isset($_POST["submit"])) {         $pattern = $_POST["pattern"];         if (get_magic_quotes_gpc()) {             $pattern = stripslashes($pattern);         }         $subject = $_POST["subject"];         @$found = preg_match($pattern, $subject);         if ($found) {             $msg = '<font color="green">Matches</font>';         } else {             $msg = '<font color="red">No match</font>';         }     }     showContent($title, $msg, $pattern, $subject); } // Display the content of a page function showContent($title, $msg, $pattern, $subject) {     echo<<<HTML <html> <head><title>$title</title></head> <body> <h1>$title</h1> <p>Enter a regular expression in <b>Pattern</b> <br>and the string to search in <b>Subject</b> <br>and then press the <b>Test</b> button.</p> <form method="POST" action="regexer.php"> <p>Pattern: <input type="text" name="pattern" value="$pattern" size="40"> <p>Subject: <input type="text" name="subject" value="$subject" size="40"> <p>$msg</p> <p><input type="submit" name="submit" value="Test"></form> </body> </html> HTML; } ?> ^ top 14.1.5: Summary In this section we looked at the language of regular expressions PHP supports many functions useful for use with regular expressions The most useful function for verifying user input is preg_match() We looked at how to match substrings using the form: /characters/ In addition, we looked at using the i and g flags to ignore case and perform global searches Then we looked at how to define character patterns using techniques such as: Character boundaries: ^, $, \b, \B Character types: \d, \D, \w, \W, \s, \S Character classes: /[a-zA-Z0-9_.-]/ Repeating characters: *, +, ?, {n}, {n,}, {n,m} Escape sequences (using a backslash: '\') Alternate patterns: /Dav(e|id)/ Grouping: /(\b\w+\b)\s+\1/ We learned about the functions associated with regular expressions so we can apply a regular expression to a number of tasks Regular expressions are very powerful -- but can be almost unreadable You must build them carefully by starting with simple expressions that work Refine and test your regular expression incrementally Build a regular expression one piece at a time and test each addition as you go Check Yourself What are regular expressions used for? What PHP function do you use to use a regular expression for verification? What can you do with a regular expression once you create one? What is a regular expression to match the first occurrence of "abc"? What is the regular expression to match every occurrence of the substring "abc" in a string, regardless of case? What is a regular expression for matching a social security number, which has nine digits? What is a regular expression for matching either "apple", "banana" or "orange"? Which RegExp method returns true if a regular expression is found in a string? Which String method returns the index of a regular expression match in a string? Which String method returns a string with all regular expression matches replaced with a specified substring? What is the first step to take when developing a regular expression? Why is setting anchors a good idea when developing a regular expression? ^ top 14.2: Scripting the Internet Learner Outcomes At the end of the lesson the student will be able to: Send email from PHP scripts Work with URLs Read and parse web pages ^ top 14.2.1: Sending Email Sometimes you want to send email: New password Order confirmation Survey results PHP provides a function called mail() that sends e-mail via SMTP Basic Syntax bool mail(toAddress, subject, message); toAddress: destination address of the e-mail subject: subject line of the email message: text of the email message For Example <?php $to = "someone@somewhere.com"; $subject = "Today's Wisdom"; $message = " A Person Who Asks A Question Is A Fool For Five Minutes. A Person Who Doesn't Is A Fool Forever"; echo mail($to, $subject, $message); ?> Security Considerations Do not use a web form for the toAddress Also, do not read a form variable for the toAddress This would let anyone use your mail server to send anything More Information mail: PHP function documentation ^ top 14.2.2: Verifying Network Information Sometimes you need to verify network information For example, you want to verify that an email address or URL is valid With PHP, you can look up hostnames, IP address and MX records An MX record is short for mail exchange record MX records are stored at the DNS and are looked up like a hostname If no MX record exists, there is nowhere for the email to go There can be more than one MX record, so the function getmxrr() returns an array Note that getmxrr() is not implemented on Windows Thus you cannot run getmxrr() on XAMPP Commonly Used Functions to Verify Network Information Function Description gethostbyaddr(ipAddress) Returns the host name of the Internet host specified by the string ipAddress. gethostbyname(hostname) Returns the IP address of the Internet host specified by the string hostname. getmxrr(hostName, mxArray) Returns an array of MX host names in mxArray from an email hostName. (Not implemented on Windows) parse_url(url) Returns from the URL string an associative array with the following indexes (if present): scheme, host, port, user, pass, path, query, and fragment. Example Checking a URL <?php $url = "http://www.edparrish.com/cis165/04s/lesson13.php"; $urlArray = parse_url($url); $host = $urlArray['host']; $ip = gethostbyname($host); if ($ip != $host) {     echo "Host for URL has a valid IP"; } else {     echo "Host for URL does not have a valid IP"; } ?> Try it! Example Checking an Email for MX Records <?php $email = "someone@totallyBogusEmailServerName.com"; $emailArray = explode('@', $email); $emailHost = $emailArray[1]; $result = getmxrr($emailHost, $mxhosts); if ($result) {     echo "MX host exist"; } else {     echo "MX host not found"; } ?> Try it! More Information Email Address Verification with PHP Network Functions ^ top 14.2.3: Reading Pages from a URL You can easily read a page from a URL $page = file_get_contents($url); Many of PHP's Filesystem functions work with Internet sources Some Functions that Read from URL's Function Description file(url) Returns an array containing the contents read from the string url, with each element of the array corresponding to a line in the file. file_get_contents(url) Returns a string containing the contents read from the string url. Note: Needs PHP version 4.3 or later. Example Script to Read From a URL <?php $url = "http://www.edparrish.com/index.html"; $page = file_get_contents($url); echo $page; ?> Try it! ^ top 14.2.4: Parsing a Web Page You can use information from other parts of the web in your own pages This is known as Web scraping In general, the steps you follow are: Find an original source URL Read the information from the URL Parse (extract) the data you want to use Finding the information might involve some detective work We looked how to read the information found in the previous section To parse the information, you often use regular expressions Function preg_match() allows you to include an extra parameter for matches to the pattern Syntax int preg_match(string pattern, string subject, array matches) pattern: regular expression pattern subject: the string to search for pattern matches matches: optional argument that is filled with the results of search Example Script to Parse a URL <?php $symbol = "AMZN"; $url = "http://www.amex.com/equities/listCmp/"       ."EqLCDetQuote.jsp?Product_Symbol=$symbol"; $page = file_get_contents($url); $pattern = "/\\\$[0-9]+\\.[0-9]+/i"; if (preg_match($pattern, $page, $matches)) {     echo "$symbol last sold at: ";     echo $matches[0]."\n"; } else {     echo "No quote available\n"; } echo "<br>Information retrieved from:<br>\n"     ."<a href=\"$url\">$url</a><br>\n"     ."on ".(date('l jS F Y g:i a T'))."\n"; ?> Try it! ^ top 14.2.5: Uploading Files Most browsers let you upload files using the POST method PHP is capable of receiving and processing uploaded files An Upload Form The following HTML creates a file upload form <form action="upload.php" method="post" enctype="multipart/form-data"> <input type="hidden" name="MAX_FILE_SIZE" value="15000"> <br>Type (or browse to) Filename:<br> <input type="file" name="uploadFile"><br> <input type="submit" value="Upload File"> </form> The attribute enctype="multipart/form-data" is needed to load files into PHPs $_FILES superglobal array The optional hidden field MAX_FILE_SIZE tells the browser the maximum file size to upload The MAX_FILE_SIZE field must be placed before the file field This field can be ignored by the browser and is easy to circumvent Thus you will need to verify the value in your script as well The file field creates the upload form element <input type="file" name="uploadFile"> Processing the Uploaded File The following is a minimal script to process an uploaded file <?php define('UPLOAD_DIR', 'uploads/'); $tmp_name = $_FILES['uploadFile']['tmp_name']; $name = UPLOAD_DIR.$_FILES['uploadFile']['name']; move_uploaded_file($tmp_name, $name); ?> PHP first places the uploaded file in a temporary directory using a temporary name Your code should move the file to its permanent location before the script finishes processing move_uploaded_file($tmp_name, $name); PHP stores all the uploaded file information in the $_FILES array Each file has an its own array of information in the $_FILES array Thus, you need to specify both the file name and data element to retrieve the value $tmp_name = $_FILES["uploadFile"]["tmp_name"]; Explanations of all the available data values are listed in the documentation for Handling file uploads Error Checking and Validation We need to both validate the file upload and check for errors First of all, you should require a user to authenticate before uploading files That way you can keep records of anyone atacking your uploading system Note that the move_uploaded_file() checks to ensure that the file is a valid upload file If any error occurs, then the file is not moved Thus you can check for errors easily with with the following code <?php define('UPLOAD_DIR', 'uploads/'); $tmp_name = $_FILES['uploadFile']['tmp_name']; $name = UPLOAD_DIR.$_FILES['uploadFile']['name']; if (move_uploaded_file($tmp_name, $name)) {     echo "File is valid, and was successfully uploaded.\n"; } else {     echo "Possible file upload attack!\n"; } echo 'Here is some more debugging info:'; echo '<pre>'; print_r($_FILES); echo "</pre>"; $error = array(    0=>"There is no error, the file uploaded successfully",    1=>"The uploaded file exceeds the upload_max_filesize    directive in php.ini",    2=>"The uploaded file exceeds the MAX_FILE_SIZE directive    that was specified in the HTML form",    3=>"The uploaded file was only partially uploaded",    4=>"No file was uploaded",    6=>"Missing a temporary folder" ); echo $error[$_FILES['uploadFile']['error']]; ?> Part of the information returned about the file is an error code $errorCode = $_FILES['uploadFile']['error']; Error codes are explained in: Error Messages Explained These codes can help you to check for user error and other problems You can use the other file information to check file types and sizes as well The following code shows how to check file type and size before uploading <?php define('UPLOAD_DIR', 'uploads/'); if (($_FILES["uploadFile"]["type"] == "image/gif") &&         ($_FILES["uploadFile"]["size"] < 15000)) {     $tmp_name = $_FILES['uploadFile']['tmp_name'];     $name = UPLOAD_DIR.$_FILES['uploadFile']['name'];     if (move_uploaded_file($tmp_name, $name)) {         echo "File is valid and was successfully uploaded.\n";     } else {         echo "Possible file upload attack!\n";     }     echo 'Here is some debugging info you can remove later';     echo '<pre>';     print_r($_FILES);     echo "</pre>";     $error = array(        0=>"There is no error, the file uploaded successfully",        1=>"The uploaded file exceeds the upload_max_filesize        directive in php.ini",        2=>"The uploaded file exceeds the MAX_FILE_SIZE directive        that was specified in the HTML form",        3=>"The uploaded file was only partially uploaded",        4=>"No file was uploaded",        6=>"Missing a temporary folder"     );     echo $error[$_FILES['uploadFile']['error']]; } else {     echo "Sorry, we only accept .GIF images under 15Kb."; } ?> More Information Handling file uploads Uploading Files To MySQL Database Working with Forms in PHP, Part 2 Large File Uploads PHP has a limit on file upload sizes -- usually about 2 MB You can change this limit in the php.ini file Also, the web server may limit the amount of information processed during one POST operation Sometimes this limit is as low as 512 KB To upload large file sizes, you may need another solution like Java or Perl More info: PHP Upload Configuration Includes links to other solutions like Applets and Perl scripts ^ top 14.2.6: Summary PHP has numerous functions for using the Internet PHP provides a funtion called mail() that sends e-mail via SMTP Function parse_url() parses a URL and returns its various parts You can use PHP functions to verify user-supplied information gethostbyname(): returns the IP address of a host, if found getmxrr(): returns the MX records for an email host, if found Also, you can read entire pages off the web: $page = file_get_contents($url); Once you read the page, you can use regular expressions to extract information preg_match($pattern, $page, $matches); The information extracted is returned in the $matches array echo $matches[0]; Browsers allow you to upload files using HTML forms PHP can recieve and process the uploaded files You should require a user to authenticate before uploading files That way you can keep records of anyone atacking your uploading system Also, you need to both validate the file upload and check for errors ^ top Exercise 14.2 Modify the following script to extract information from a web page of your choosing. <?php $symbol = "AMZN"; $url = "http://www.amex.com/equities/listCmp/"       ."EqLCDetQuote.jsp?Product_Symbol=$symbol"; $lines = file($url); $price = ""; $pattern = "/\\\$[0-9]+\\.[0-9]+/i"; foreach ($lines as $line) {     if (preg_match($pattern, $line, $matches)) {         $price = $matches[0];         break;     } } if ($price) {     echo "$symbol last sold at: $price\n"; } else {     echo "No quote available\n"; } echo "<br>Information retrieved from:<br>\n"     ."<a href=\"$url\">$url</a><br>\n"     ."on ".(date('l jS F Y g:i a T'))."\n"; ?> ^ top 14.3: Finishing the Course Learner Outcomes At the end of the lesson the student will be able to: Discuss the final preparation for the project presentation Advise the instructor on how to improve future courses ^ top 14.3.1: About the Final Project Presentation Important Final Project Information Date and Time: 4:00 pm-6:50 pm Tuesday, June 1 Location: Room 2501 (regular classroom) Mandatory attendance for all students Before the Presentation Submit your project to Blackboard before the presentation: Bring a written report on paper to give to the instructor before the presentation During the Presentation The presentation should have the following: Your name and your project's name A brief introduction describing the purpose of your project A demonstration and discussion of the user interface including: Entry page Page layout Navigation features A demonstration of a multi-form sequence where you pass information from one page to another A demonstration of user-input error handling: Checking of form input for errors Highlighting of errors so users easily see them Explanation to user of how to correct errors Retention of prior entries on error (except passwords) A discussion or demonstration of user authentication: How the database is used for authentication How passwords are encrypted in database A discussion or demonstration of security features: How data types are checked before insertion into a database How data sizes are checked before insertion into a database How taint checking of special characters is implemented (e.g. '"$#) How special symbols and spaces do not cause database errors A discussion or demonstration of cool features: Point them out so we can all appreciate them Feel free to display your written report during the presentation Keep the presentation to 10 minutes or less After the Presentation Feel free to leave (or stay) after your presentation You can present to the instructor alone after the other presentations are through ^ top 14.3.2: Lecture Finale During the semester we have covered many topics and learned at least two languages: SQL and PHP With this knowledge you can develop professional-looking database-driven Web sites Your project will allow you to demonstrate what you have learned: There is no substitute for a working application! I hope that everyone has enjoyed taking the course as much as I have enjoyed presenting it I am always open to suggestions for improving the course ^ top Wrap Up Due Next: Final Project Report and Presentation (6/1/10) When class is over, please shut down your computer if it is on ^ top Home | Blackboard | Syllabus | Expectations | Schedule Project | Help | FAQ's | HowTo's | Links Last Updated: May 24 2010 @19:16:49