Brief Summary: This early phase I trial studies how well pembrolizumab works in treating patients with gastroesophageal adenocarcinoma that has spread to other places or cannot be removed by surgery. Immunotherapy with monoclonal antibodies, such as pembrolizumab, may help the body's immune system attack the cancer, and may interfere with the ability of tumor cells to grow and spread.

Detailed Description: PRIMARY OBJECTIVES: I. To estimate the overall response rate (ORR) of intravenous administration of pembrolizumab by Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 in adult subjects with advanced (unresectable and/or metastatic) and previously treated gastroesophageal adenocarcinoma. SECONDARY OBJECTIVES: I. To evaluate the duration of response (DOR), disease control rate (DCR), time to progression (TTP), progression-free survival (PFS), and overall survival (OS) using RECIST 1.1 and immune-related (ir)RECIST criteria. II. To perform exploratory biomarkers to study the correlation between immunological and molecular changes in tumor tissues and peripheral blood with clinical outcomes (DOR, DCR, TTP, PFS, and OS) using RECIST 1.1, and irRECIST rate of adverse events. III. To determine the safety and tolerability of intravenous administration of pembrolizumab or pembrolizumab plus ramucirumab in adult subjects with advanced (unresectable and/or metastatic) and previously treated gastroesophageal adenocarcinoma. EXPLORATORY OBJECTIVES: I. To explore the association between PD-L1 expression by immunohistochemistry, shed PD-L1 levels, somatic gene expression profiling and antitumor efficacy of pembrolizumab based on RECIST 1.1 imaging criteria as well as overall survival. II. To explore the relationship between genomic variation and response to the treatment administered. III. Tissue and blood immune monitoring will be conducted through our immune platform group as detailed per the biomarker section based on 3 biopsies done at the following time points: 1) pre-treatment, 2) 6 weeks after therapy, and 3) and an optional biopsy upon progression. OUTLINE: Patients receive pembrolizumab intravenously (IV) over 30 minutes on day 1. Treatment repeats every 21 days for up to 35 cycles in the absence of disease progression or unacceptable toxicity. After completion of study treatment, patients are followed up at 30 days and then every 12 weeks thereafter.