Fifth Benelux Bioinformatics Conference, Liège, 14­15 December 2009

Dynamic Treatment Regimes using Reinforcement Learning: a  Cautious Generalization Approach Raphael Fonteneau, Susan Murphy, Louis Wehenkel, Damien Ernst University of Liège, University of Michigan

The   treatment   of   chronic­like   illnesses   such   has   HIV   infection,   cancer   or   chronic  depression   implies   long­lasting   treatments   that   can   be   associated   with   low   quality  outcome,   painful   side   effects   and   expensive   costs.   To   enhance   these   treatments,  clinicians often adopt what we call Dynamic Treatment Regimes (DTRs). DTRs are  sets of sequential decision rules defining what actions should be taken at a specific  instant to treat a patient based on information observed up to that instant. Ideally,  DTRs should lead to treatments which result in the most favorable clinical outcome  possible.   Since   a   few   years,   a   growing   research   community   is   working   on   the  development   of   formal   methods   (mainly   issued   from   mathematics,   statistics   and  control theory) that allow to infer from clinical data high­quality DTRs. We propose in this framework an algorithm of quadratic complexity that infer from  clinical   data   a   sequence   of   treatment   actions.   The   algorithm   (called   CGRL   for  Cautious   Generalization   for   Reinforcement   Learning)   has   cautious   generalization  properties, i.e. it avoids taking treatment actions for which the sample of clinical data  is   too   sparse   to   make   safe   generalization.   The   algorithm   also   has   consistency  properties, which means that when the sparsity of the set of clinical data decreases to  zeros,   the   inferred   sequence   of   treatment   actions   is   actually   optimal.   Moreover,   in  some favorable cases, some tight performance guarantees on the inferred sequence of  treatment actions can be computed. The algorithm is illustrated using some simulated  data dealing with the HIV infection.

Dynamic Treatment Regimes using Reinforcement ...

Fifth Benelux Bioinformatics Conference, Liège, 1415 December 2009. Dynamic ... clinicians often adopt what we call Dynamic Treatment Regimes (DTRs).

159KB Sizes 0 Downloads 265 Views

Recommend Documents

Dynamic Treatment Regimes using Reinforcement ...
Dec 15, 2009 - Raphael Fonteneau, Susan Murphy, Louis Wehenkel, Damien Ernst. University of Liège, University of Michigan. The treatment of chroniclike illnesses such has HIV infection, cancer or chronic depression implies longlasting treatments that

Variable selection for dynamic treatment regimes: a ... - ORBi
will score each attribute by estimating the variance reduction it can be associ- ated with by propagating the training sample over the different tree structures ...

Variable selection for Dynamic Treatment Regimes (DTR)
Jul 1, 2008 - University of Liège – Montefiore Institute. Variable selection for ... Department of Electrical Engineering and Computer Science. University of .... (3) Rerun the fitted Q iteration algorithm on the ''best attributes''. S xi. = ∑.

Variable selection for Dynamic Treatment Regimes (DTR)
Department of Electrical Engineering and Computer Science. University of Liège. 27th Benelux Meeting on Systems and Control,. Heeze, The Netherlands ...

Variable selection for dynamic treatment regimes: a ... - ORBi
Nowadays, many diseases as for example HIV/AIDS, cancer, inflammatory ... ical data. This problem has been vastly studied in. Reinforcement Learning (RL), a subfield of machine learning (see e.g., (Ernst et al., 2005)). Its application to the DTR pro

Variable selection for dynamic treatment regimes: a ... - ORBi
n-dimensional space X of clinical indicators, ut is an element of the action space. (representing treatments taken by the patient in the time interval [t, t + 1]), and xt+1 is the state at the subsequent time-step. We further suppose that the respons

Variable selection for Dynamic Treatment Regimes (DTR)
University of Liège – Montefiore Institute. Problem formulation (I). ○ This problem can be seen has a discretetime problem: x t+1. = f (x t. , u t. , w t. , t). ○ State: x t. X (assimilated to the state of the patient). ○ Actions: u t. U. â—

Workstation Capacity Tuning using Reinforcement ...
Perl and C++ APIs. It relies on a ... The user uses the Command Line Interface (CLI) or API ...... the lower hierarchy machines are grouped and managed by.

Small-sample Reinforcement Learning - Improving Policies Using ...
Small-sample Reinforcement Learning - Improving Policies Using Synthetic Data - preprint.pdf. Small-sample Reinforcement Learning - Improving Policies ...

Dynamic Discrete Choice and Dynamic Treatment Effects
Aug 3, 2006 - +1-773-702-0634, Fax: +1-773-702-8490, E-mail: [email protected]. ... tion, stopping schooling, opening a store, conducting an advertising campaign at a ...... (We recover the intercepts through the assumption E (U(t)) = 0.).

Security Regimes
that security regimes, with their call for mutual restraint and limitations on unilateral .... made subordinate; wherever His voice can be heard, it will be raised to discourage ..... At the May 1972 summit conference in Moscow, the U.S. and Soviet.

Bounding Average Treatment Effects using Linear Programming
Mar 13, 2013 - Outcome - College degree of child i : yi (.) ... Observed Treatment: Observed mother's college zi ∈ {0,1} .... Pencil and Paper vs Computer?

Chronic temporomandibular pain treatment using sodium diclofenac ...
Sandra Sato*; Murillo Sucena Pita**; Cássio do Nascimento*. & Vinícius Pedrazzi***. VAROLI, F. L.; SATO, S. ... After, were made a flat, full-covered and rigid. occlusal splint for each volunteer. They had ... Page 3 of 6. Main menu. Displaying Chr

Exploratory PerFormance Evaluation using dynamic ...
plete 'active' sub-models that serve, e.g., as embedded strategies that control the .... faces, their signature, and that these interfaces behave as described above ...

Dynamic Attack Mitigation using SDN
Abstract—Security threats in the Internet have been ever increasing, in number, type and means used for attacks. In the face of large-scale attacks, such as DDoS attacks, networks take unacceptable time to respond and mitigate the attacks, resultin

Coexistence Mechanism Using Dynamic Fragmentation ...
what proposed in [9] requires too much complexity to obtain the optimal solution ..... PDA is connected to a laptop via Wi-Fi at rate 11Mbps, and concurrently a ...

Automatic speaker recognition using dynamic Bayesian network ...
This paper presents a novel approach to automatic speaker recognition using dynamic Bayesian network (DBN). DBNs have a precise and well-understand ...

reinforcement survey.pdf
tell us the things your child likes so we will have an idea of what they will want to work. for at school. ... q apple slices. q juice. q chips ... reinforcement survey.pdf.

reinforcement survey.pdf
for at school. Toys Does your child like to play with toys? Yes No. Activities. Tokens Food. Is your child allowed to be reinforced with food? Yes No. q cars. q puzzles. q dolls. q blocks. q Legos. q character toys. What characters? ______. ______. _

Core, Periphery, Exchange Rate Regimes, and Globalization
The key unifying theme for both demarcations as pointed out by our ...... Lessons from a Austro-Hungarian Experiment (1896-1914)” WP CESifo, University of.

Evaluating Treatment Protocols using Data Combination
Nov 22, 2012 - endpoints of the study included death and myocardial infarction (heart attack), and ... endpoints included evaluation of angina and quality of life.

Reinforcement Learning Trees
Feb 8, 2014 - size is small and prevents the effect of strong variables from being fully explored. Due to these ..... muting, which is suitable for most situations), and 50% ·|P\Pd ..... illustration of how this greedy splitting works. When there ar

Methods of treatment using a gastric retained losartan dosage
Aug 21, 2003 - is commercially available in 25 mg, 50 mg and 100 mg tablet dosage forms. ... of losartan's major side effect, diZZiness (McIntyre, et al., supra).