Glassbox testing

• use code directly to guide design of test cases
• called path complete if every potential path through code is tested at least once
• What are some drawbacks of this type of testing?
  • can go through loops arbitarily many times
  • even in the absence of loops, the number of paths can be huge. Show a stacked-diamonds shaped control flow.
• Guidelines
  • Branches
    • Exercise all parts of a conditional
  • For loops
    • Loop not entered
    • Body of loop executed exactly once
    • Body of loop executed more than once
  • While loops
    • Same as for loops
    • Cases that catch all ways to exit the loop

Path-complete testing is not necessarily enough

def abs(x):
  """ Assumes x is an int
      Returns x if x>=0 and –x otherwise """
  if x < -1:
    return –x
  else:
    return x

• A path-complete test suite could miss a bug
• Path-complete test suite: -2 and 2
• But abs(-1) incorrectly returns -1
• Should still test boundary cases

Debugging

• Steep learning curve
• Goal is to have a bug-free program
• Tools
  • Python tutor
  • print statement
  • use your brain, be systematic in your hunt

Print statements

• good way to test hypothesis
• when to print
  • enter function
  • parameters
  • function results
• use bisection method
  • put print halfway in code
  • decide where bug may be depending on values

Debugging steps

• study program code
  • don't ask what is wrong
  • ask how did I get the unexpected result
  • is it part of a family
• scientific method
  • study available data
  • form hypothesis
  • repeatable experiments
  • pick simplest (or smallest) input to reproduce the bug

Error messages - easy Consider the following program

test = [1, 2, 3]

• Trying to access beyond the limits of a list

  test[4] # IndexError
  

• Trying to convert an inappropriate type

  int(test)  #TypeError
  

• Referencing a non-existent variable

  a  #NameError
  

• Mixing data types without appropriate coercion

  '3'/4  #TypeError
  

• Forgetting to close parenthesis, quotation, etc.

  a = len([1, 2, 3]
  print(a)  #Syntax Error
  

Logic Errors - Hard

• think before writing new code
• draw pictures, take a break
• explain the code to
  • someone else
  • to yourself (if nobody wants to listen to you)

Exceptions and Assertions

• What happens when procedure execution hits an unexpected condition?
• Get an exception... to what was expected
  • trying to access beyond list limits

    test = [1, 7, 4]
    test[4] # IndexError
    

• trying to convert an inappropriate type

  int(test) # TypeError
  

• referencing a non-existing variable

  a  # NameError
  

• mixing datatypes without coercion

  'a'/4  # TypeError
  

Other examples of exceptions: already seen common datatypes

• SyntaxError: Python can't parse program
• NameError: Local or global name not found
• AttributeError: Attribute reference fails
• TypeError: Operand does not have correct type
• ValueError: Operand type okay, but value is illegal
• IOError: IO system reports malfunction (e.g., file not found)

Dealing with exceptions

• Python code can provide handlers for exceptions

  try:
    a = int(input("Tell me one number:"))
    b = int(input("Tell me another number:"))
    print(a/b)
  except:
    print("Bug in user input.")
  

• Exceptions raised by any statement in body of try are handled by the except statement and execution continues with the body of the except statement.

You can have separate except clauses to deal with a particular type of exception

try:
  a = int(input("Tell me one number:"))
  b = int(input("Tell me another number:"))
  print(a/b)
except ValueError:
  print("Could not convert to a number.")  # only executes if ValueError comes up
except ZeroDivisionError:
  print("Can't divide by zero.")  # only executes if ZeroDivisionError comes up
except:  # for all other errors
  print("Something else went wrong.")

What to do when you encounter an exception?

• fail silently:
  • substitute default values or just continue
  • bad idea! user gets no warning
• return an "error" value
  • Examples: -1, empty string, None, etc.
  • What value to choose?
  • Complicates code having to check for a special value. A good value should keep the code succinct and intuitive, e.g., return zero on success and negative integers on error, where the magnitude of a negative number indicates the reason for error.
• stop execution, signal error condition
  • In Python: raise an exception

    raise Exception("descriptive string")
    

Exceptions as control flow

• The raise keyword can be used to indicate an exception (with its name and description) at runtime.
• Application:
  • When an error occurs, do not return special values, instead raise an exception, and add an except block at the very end.
  • Relay an exception after potentially changing its name (e.g., a common name for many exceptions depending on the context) and description.

raise <exceptionName>(<arguments>)

raise ValueError("something is wrong")  #The string description is optional