A Fast Computer Intrusion Detection Algorithm Based on Hypothesis Testing of Command Transition Probabilities

William DuMouchel, Matthias Schonlau

Abstract: This statistical method compares in real time the sequence of commands given by each user to a profile of that user's past behavior. We use the Fisher score statistic to test the null hypothesis that the observed command transition probabilities come from a profiled transition matrix. The alternative hypothesis is formed from a principal components analysis of historical differences between the transition probabilities of all other users and those of the user being tested. The calculations can be structured so that only a few dozen arithmetic operations are needed to update an online test statistic after each submitted command. The theoretical statistical properties of the test, such as false positive and false negative rates, are computable under the assumptions of the markov process model. Based on a population of 45 research users on a single computer, test data from each user are used to challenge the profile of every user. The test had sufficient statistical power to successfully discriminate between almost every pair of users based on a sample size equivalent to a single day's usage of an average user.