Plotnikova I.A1, Troitskaya LA2, Zelentsova V.L1, Stepina D.A1., Shtanova A.A 1

1Department of Childhood Diseases, Faculty of Treatment and Prevention Ural State Medical University, Yekaterinburg, Russian Federation. Repina str., Yekaterinburg, Sverdlovsk region, Russian Federation 620028
2Pirogov Russian National Research Medical University of the Ministry of Health of the Russian Federation, Department of Nervous Diseases, Moscow

*Corresponding author

*Plotnikova I.A,Department of Childhood Diseases, Faculty of Treatment and Prevention Ural State Medical
University, Yekaterinburg, Russian Federation. 3 Repina str., Yekaterinburg, Sverdlovsk region, Russian
Federation 620028


The article presents a clinical case of focal epilepsy with a status course of seizures associated with a genetic mutation in exon 1 of the PNPO gene, which led to pyridoxamine-5'-phosphate oxidase deficiency. The diagnosis was made late due to the misinterpretation of symptoms, which complicated the course of the disease. Despite the fact that the first symptoms in the form of seizures appeared at the age of 1 month, only at the age of 5 the diagnosis was verified by doing targeted DNA sequencing. At the moment, the patient is receiving substitution therapy in the form of pyridoxal phosphate 300 mg/day, which enabled unstable clinical remission. Right now, it is impossible to achieve complete control over the convulsive syndrome without a strict diet: dairy-free, meat-free, egg-free and low-protein fat-free food. Currently, further search for treatment methods continues to improve the patient's quality of life and ensure stable remission. A detailed analysis was given for further genetic verification based on the amino acid profile of the patient, and the rehabilitation potential was determined based on topical neuropsychological diagnostics performed on a non-verbal child.

Key words: focal epilepsy; Pyridoxal 5′-phosphate; vitamin B6; PNPO; vitamin B6-dependent epilepsy, neuropsychological diagnostics.


Vitamin B6-dependent epilepsies are aheterogeneous group of autosomal recessive diseases that are caused by mutations of five different genes involved in vitamin B6 metabolism [1]. Vitamin B6 is present in many forms in the human diet, but only pyridoxal-5 -phosphate (PLP) plays a vital role in the metabolism of a number of neurotransmitters, especially the inhibitory mediator gamma-aminobutyric acid. Code errors leading to a lack of pyridoxal-5'-phosphate manifest as B6-dependent epilepsy, including pyridoxamine-5-phosphate oxidase (PNPO) deficiency, which affects the synthesis and recycling of pyridoxal-5'-phosphate [2,3]. Neonatal manifestation in the form of acute encephalopathy with biphasic epileptic seizures (or status epilepticus) is the main symptom of the disease. The first phase (early attacks) is accompanied by fever and a temporary recovery of consciousness and the development; the second phase is a global cognitive dysfunction (late attacks).

Resistance to traditional antiepileptic therapy requires patient's lifelong treatment by pharmacological doses of vitamin B6 in the form of pyridoxine (PN) or a biologically active form of pyridoxal-5’-phosphate [1,4].

Case reports of PLP deficiency, verified not only clinically, but also by exome sequencing, are quite rare as well as the methods for studying molecular markers of alpha-aminoadipic semialdehyde and pipecolic acid in body fluids [5–7]. The complexity of diagnosis is caused by multiple disorders in newborns, especially in case of a slow and incomplete response to pyridoxine [8].

Recent studies have shown that the main enzyme defect in pyridoxine-dependent epilepsy is caused by alpha-aminoadipic acid semialdehyde dehydrogenase in the pathway of cerebral lysine degradation. The accumulating compound, alpha-aminoadipine semialdehyde (alpha-AASA), is in equilibrium with delta-1-piperidine-6-carboxylate (P6C). P6C inactivates pyridoxal-5’-phosphate, causing severe cerebral insufficiency. Although treatment of pyridoxal 5'-phosphate deficiency can successfully control seizures, most patients develop some degree of disability, regardless of early diagnosis and treatment. Very few patients with normal intelligence have been reported [7].

Objective: to analyze the course of epilepsy with pyridoxamin-5’-phosphate oxidase deficiency in an 8-year-old patient with diagnosis verification by clinical exome sequencing.

Materials and Methods of Research.

The analysis of primary medical documentation from 2013 to 2021 of a patient born in 2013 was performed. We reviewed the materials on the topic using PubMed search engines for the period 2014-2021, correlation of literature data with a specific clinical case.

Research results and their discussion.

A clinical case

Girl, 8 years old, was born from IV pregnancy of a woman with a burdened obstetric history. At the age of 1 month, tonic-clonic convulsions were first noted during sleep: gaze adversion to the left, lasting 30 seconds - 1 minute; afterwards there was up to 4 seizures per day, daily. At the age of 1 year, she was hospitalized 4 times on an emergency basis for convulsive seizures. The child was observed by a neurologist-epileptologist with a diagnosis of perinatal damage to the central nervous system, recovery period. Valproic acid was prescribed at a dosage of 50 mg/kg per day, oxcarbazepine 300 mg/day, without an effect of therapy. At the age of 2 years, she was hospitalized three times in the intensive care unit due to the status course of an epileptic seizure with a rise in temperature to febrile numbers. Neurological diagnosis at that time was: symptomatic epilepsy with complex partial seizures, status course of generalized convulsive seizures. On electroencephalography (EEG): moderate diffuse changes in the bioelectric activity (BEA) of the brain in a disorganized type. The patient's condition worsened. At the age of 3 years, she was observed in the State Autonomous Healthcare Institution of the Sverdlovsk Region "Children's City Clinical Hospital No. 9, Yekaterinburg" with the same diagnosis; the dose of oxcarbazepine was increased to 500 mg/day, valproic acid to 300 mg/day with no significant clinical effect. At the age of 4 years, she was hospitalized three times in the intensive care unit about epileptic seizures, without the effect of anticonvulsant therapy. Concomitant diseases at age 4 were: severe osteoporosis of the visible parts of the skeleton; pathological compression fracture of the body Th11; hepatomegaly; moderate expansion of the common hepatic, common bile ducts; enlargement of the gallbladder; a pronounced increase in the size of the kidneys, pancreas; diffuse changes in the parenchyma of the kidneys, a single cyst of the right kidney; unspecified form of caries; chronic gingivitis. Computed tomography of the abdominal aorta and its branches showed no evidence of hepatic artery stenosis. Autonomic dysfunction of the sinus node was noted: sinus arrhythmia with episodes of bradycardia. There were also small anomalies in the development of the heart: a functioning foramen ovale, additional chords of the cavity of the left ventricle.

DNA sequencing was carried out in 2017. Genetic mutations that were identified are described in patients with epilepsy associated with pyridoxamine 5'-phosphate oxidase deficiency and, based on the totality of information, regarded as pathogenic - a mutation in exon 1 of the PNPO gene (chr17: 46019139A> T, rs370243877), leading to amino acid replacement at position 33 of the protein (p.Asp33Val, NM-018129.3, mutation frequency in the ExAC control sample 0.0235%); as probably pathogenic - a previously undescribed heterozygous mutation in intron 3 of the PNPO gene (chr17:46022086G>A, rs766037058), leading to disruption of the splicing site and synthesis of the full-length protein (c.363+5G>A, NM_018129.3, OMIM: 610090, the value of the algorithm for predicting its influence on the function of AdaBoost splicing sites is 1.000).

A heterozygous mutation was also found in exon 4 of the EARS2 gene (chr16:23546678A>T), leading to a premature translation termination site at codon 163 (p.Tyr163Ter, NM_001083614.1). Such mutations have been described in patients with combined oxidative phosphorylation deficiency type 12 (OMIM: 614924). In this case (when no second mutation in the gene is detected), the result is regarded as an option with uncertain clinical significance, however, the mutation may be related to the phenotype. The parents did not undergo a genetic examination.

Prescribed treatment was: pyridoxine hydrochloride intramuscularly, then - pyridoxal phosphate at the rate of 10-50 mg / kg /day. On the 7th day after the start of treatment, the patient's consciousness was assessed as clear, she was able to sit up independently and stand with support. Her seizures stopped, appetite improved, during rehabilitation positive dynamics in neuropsychic development was noted with an expansion of the range of motor activity, the appearance of gaming activity, emotions and attempts to pronounce individual sounds.

At the age of 5 years 1 month there was a new epileptic seizure. The dose of pyridoxal phosphate was increased to 600 mg/day, convulsive attacks stopped. Concomitant diseases at age 5 were perianal dermatitis, vulvitis, continuously recurrent leukocyturia. Subsequent courses of medical rehabilitation was prescribed with positive dynamics.

In 2019, hyperkinesis (blinking), tremor, restlessness reappeared; in the summer were tonic-clonic seizures with vocalization, lasting 15-20 minutes and the status course of an attack, operculations, loss of appetite. By the end of the year, there was constant nausea and a gag reflex at the sight of food, vomiting with yellow mucus and a sour smell once every 5-7 days, accompanied by febrile fever, the smell of "rotten cheese" from the scalp and excrements during attacks. Motor clonic seizures appeared with a frequency of once every 1-2 months, symmetrical chill-like tremor - up to 3-5 times a day. Periodic episodes of psychomotor agitation, stereotyped movements were also noted.

Neurological status. There are bradypsychia, delayed psycho-motor development, coordination disorder. Patient does not pronounce words, speech is active only during the game-vocalisms, self-service skills are not formed. Autism spectrum disorders with general speech underdevelopment of level 1, psychomotor alalia were noted. Cerebral, meningeal symptoms are negative. The gait is uncertain. Cerebellar tests are negative. Cranial nerves: palpebral fissures D=S, pupils D=S, pupil reaction to light: direct D=S, consensual D=S. The volume of movement of the eyeballs is complete D=S, there is no nystagmus. The face is symmetrical D=S. There is no language deviation. Swallowing, phonation are not disturbed. Muscle tone: arms - reduced D=S, legs - normal D=S. Tendon reflexes: from the arms and legs increased D=S. There are no pathological foot signs, pelvic functions are preserved. Patient shows signs of slightly asymmetrical (with an accent on the left) motor awkwardness, reduced nutrition (Body weight 21,5 kg).

Results of instrumental and laboratory studies. The following disorders were detected on the EEG prior to the start of etiological therapy: Epileptiform activity in the form of "peak-wave" complexes in the frontal and central-temporal leads, more on the right; slowing down of activity in the temporal zone.

In the biochemical analysis of blood the level of amino acids (µmol/l) is low: alanine 119.30; glutamic acid 72.00; glycine 86.50; ornithine 22.10; proline 87.00. Activity of alanine aminotransferase is 24.9 U/l (reference values 0-29 U/l), aspartate aminotransferase - 26.4 U/l (reference values 0-48 U/l).

 Control visit. After the diagnosis was verified by exome sequencing, the patient was prescribed etiotropic therapy: pyridoxal phosphate 300 mg/day. The pre-elevated (1070 nmol/l) plasma concentration of vitamin B6 (pyridoxal-5-phosphate) normalized. EEG data - video monitoring showed moderately severe violations of BEA of the brain; the main rhythm is formed by age; registered regional slowing of the rhythm in the right central-parietal region. Epileptiform activity, clinical paroxysms, EEG patterns of epileptic seizures were not registered.

Final diagnosis: Genetic focal epilepsy due to a mutation in the PNPO gene (chr17: 46019139A> T, rs370243877). The type of attack is focal with impaired consciousness. PNPO developmental and epileptic encephalopathy. Cognitive impairment. Alalia. Motor awkwardness.

Psychological status. Diagnostics of cognitive activity showed that the girl is accessible to contact; she does not speak and comprehension of the speech is shown only in the form of understanding simple commands and simple instructions for the task. The child's object-sensory activity is carried out 100% through visual perception and shape perception, the perception of size is developed by 50%, spatial perception - 12%, color perception is completely absent. The insufficiency of these afferentations is a consequence of the decrease in the “zone of actual development”, which may be attributed to pedagogical neglect. In the motor sphere, gross motor skills are fully formed, fine motor skills are developed by 54%, objective activity is formed by 9%, taking into account the skills of game and constructive praxis, speech function is developed by 25%, self-service skills - by 60%, socialization – by 40%. Psychological diagnostics of the state of higher mental functions was carried out by depicting the structural and functional features of the brain, as a result of which topical insufficiency of brain areas was revealed. Figure 1 shows the level of formation of brain zones.

Figure 1: The degree of formation of brain departments that implement sensory and motor skills.

Despite the pronounced cognitive deficit in the child, the implementation of the program of psychological rehabilitation may expand the "zone of actual development" in the structure of the sensory, subject and pedagogical profile (since there are preserved components of cognitive activity)


Patient’s clinical diagnosis was established only at the age of 5 years, based on clinical manifestations and exome sequencing. The primal reduction of the dose of pyridoxal-5'-phosphate provoked a relapse of status epilepticus and a regression of acquired cognitive skills. A subsequent increase of treatment in combination with dietary therapy provided an unstable clinical remission without further improvement in the patient's condition. Such a response to the therapy has also been demonstrated in other studies [6,7].

Although in patients with a typical course of the disease, there is a several-fold increase in the level of glycine and glutamic acid in the blood plasma [1,5–7,9], in our case there is a decrease in glycine to 86,50 µmol/l (norm: 100-400 µmol /l) and other amino acids. Hypoglycinemia is an extremely rare condition, it occurs only in severe hereditary aminoacidopathy, but in our patient, tandem mass spectrometry was performed twice (including against the background of an attack) in 2016 and did not show any data of hereditary aminoacidopathy, organic aciduria, defects β-oxidation of fatty acids. The girl has a positive reaction to the oral intake of amino acid complexes and glycine separately, therefore, additional genetic analysis can be performed for 3-phosphoglycerate dehydrogenase deficiency, the clinical manifestations of which may be encephalopathy and seizures unresponsive to anticonvulsants [10]. Symptoms of this disease can be stopped by joint intake of serine and glycine so this diet may be developed for our patient. The study of vitamin B6 metabolites in de novo serine biosynthesis by Ramos et al (2017) had one group of rats which received a pyridoxine-deficient diet, while the diet of the control group of rats contained a normal amount of pyridoxine. This study has demonstrated a decrease in serine biosynthesis in Neuro-2a cells in vitamin B6 deficient rats. The pyridoxal-5'-phosphate-dependent enzyme phosphoserine aminotransferase (PSAT, EC cannot function fully in conditions of vitamin B6 deficiency, and likely reduces the synthesis of phosphoserine and serine in animals on a pyridoxine-deficient diet. The production of glycine depends on the availability of serine and on the pyridoxal-5'-phosphate-dependent enzyme SHMT, which catalyzes part of the transformation of glycine, and the simultaneous deficiency of serine and pyridoxal-5'-phosphate can reduce its activity and lead to a decrease in the content of glycine in blood plasma [9].

Some authors reported EEG changes in patients with pyridoxine-dependent epilepsy [11]. In our patient, no clear epileptiform activity was registered either before or after the start of treatment with pyridoxal-5'-phosphate; this variant of EEG was also described by other researchers [5,6]. Changes in the brain during magnetic resonance imaging in patients with pyridoxine-dependent epilepsy may vary from normal to diffuse atrophy of the gray and white matter of the hemispheres [2]; in our case no changes were detected.

According to Plecko B. Et al., with late diagnosis stable remission after the appointment of pyridoxal-5'-phosphate is observed only in a few patients [1]. Early treatment is critical to prevent irreversible damage to the central nervous system and shows positive results [1,5,6]. Patients with pyridoxine-dependent epilepsy require lifelong supplementation with pyridoxal-5'-phosphate. Therapeutic doses of the drug vary from 15 to 30 mg/kg/day [1]. The daily requirement for vitamin B6 in infancy is 0.1–0.3 mg. Pyridoxal-5'-phosphate doses up to 500 mg/day are considered safe in children with classical vitamin B6 deficiency, but higher doses may cause reversible sensory and rare motor neuropathy [1], so total daily doses of pyridoxal-5'-phosphate, should not exceed 200-300 mg. There are no data on the optimal dose of the vitamin for long-term treatment. In experimental animals, doses of pyridoxal 5'-phosphate >50mg/kg/d induce ataxia, peripheral neuropathy, and muscle weakness; histological examination demonstrates neuronal damage with loss of myelin and degeneration of sensory fibers in peripheral nerves, dorsal columns of the spinal cord, and descending tract of the trigeminal nerve. In most cases of peripheral neuropathy, the total dose of pyridoxal 5'-phosphate is >1000 mg/day. Some children who take high concentrations of pyridoxal-5'-phosphate develop a persistent increase in transaminases with progression to cirrhosis and hepatocellular carcinoma [3]. To avoid side effects, a fixed effective dose should be used. However, studies showed that daily doses up to 1100 mg/day and 50 mg/kg/day to achieve a state without epileptic seizures did not cause any side effects when they were divided into 4–5 doses per day [12]. In our case the doses of pyridoxal-5'-phosphate less than 600 mg/day induces epileptic seizures and cognitive disfunction. Some mutations in the genes encoding of pyridoxamine-5-phosphate oxidase may require the combined treatment with pyridoxal-5'-phosphate and pyridoxine [12,13]. It is possible that such treatment will have a positive response in our patient as well.

Another interesting feature of this clinical case is an intolerance of the patient to many products: remission occurs only on a low-protein, low-fat diet with the exclusion of dairy, meat products and eggs. Similar dietary restrictions are observed in ALDH7A1 deficiency (antiquitin deficiency), which often accompanies PNPO gene mutation. In our case ALDH7A1 deficiency was excluded by exome sequencing [13,14]. However, a lysine-restricted diet can also be effective for homozygous mutations in the PNPO gene in some patients [14]. As an example of a diet, the recommendations of Koelker and Ross on glutaric aciduria type I can be used [15].

The patient also has a high content of vitamin B6 in plasma (775.0 nmol/l), which is typical response to an intake of pyridoxal-5'-phosphate (described levels of vitamin B6 in plasma: 400 nmol/l, 1060 nmol /l and 624 nmol/l) [12,18]. It is not known why some patients continue to have seizures even when taking high doses of pyridoxal-5'-phosphate, while others grow almost normally [1,7,19]. The long-term prognosis for this patient remains unclear. For our patient a clarifying genetic study with modification of pharmacological treatment and diet is required, considering that the girl does not tolerate protein hydrolysates and an unstable clinical remission only on a low-protein low-fat diet with the exclusion of dairy, meat products and eggs.


  1. DNA diagnostics using the method of sequencing of exome regions of the genome is a key method for early verification of the diagnosis of epilepsy in newborns and young children, which in combination with the therapy can improve the prognosis.
  2. The presence of heterozygous mutations in this clinical case suggests other metabolic deficits, which complicates the selection of treatment and requires additional examination of the exome.
  3. To ensure stable remission, nutritional correction is required to compensate for deficient conditions during severe elimination measures, as well as the selection of the minimum sufficient dosage of pyridoxal-5'-phosphate in combination with pyridoxine hydrochloride.
  4. Topical neuropsychological diagnostics and psychological correction based on intact higher mental functions makes the recovery of the patient possible.

Conflict of Interest: The authors of this article have confirmed that there are no conflicts of interest or financial support to report.


  1. Plecko B. Pyridoxine and pyridoxalphosphate-dependent epilepsies. In 2013. p. 1811–7. doi: 10.1016/B978-0-444-59565-2.00050-2.
  2. Akiyama T, Toda S, Kimura N, Mogami Y, Hanaoka Y, Tokorodani C, et al. Vitamin B6 in acute encephalopathy with biphasic seizures and late reduced diffusion. Brain Dev. 2020 May;42(5):402–7. doi: 10.1016/j.braindev.2020.02.002
  3. Wilson MP, Plecko B, Mills PB, Clayton PT. Disorders affecting vitamin B 6 metabolism. J Inherit Metab Dis. 2019 Jul 20;42(4):629–46. doi: 10.1002/jimd.12060.
  4. Saygitov RT, Ostrovskaya AS. Guidelines for preparation of manuscripts describing clinical cases in children: SUPPORTING THE CARE AND SCARE PRINCIPLES. Curr Pediatr. 2017 Jan 1;16(5):352–7. (in Russ) doi: 10.15690/vsp.v16i5.1798
  5. Nam SH, Kwon M-J, Lee J, Lee CG, Yu HJ, Ki C-S, et al. Clinical and genetic analysis of three Korean children with pyridoxine-dependent epilepsy. Ann Clin Lab Sci. 2012;42(1):65–72.
  6. Yang Z, Yang X, Wu Y, Wang J, Zhang Y, Xiong H, et al. Clinical Diagnosis, Treatment, and ALDH7A1 Mutations in Pyridoxine-Dependent Epilepsy in Three Chinese Infants. PLoS One. 2014 Mar 24;9(3):e92803. doi: 10.1371/journal.pone.0092803
  7. Kluger G, Blank R, Paul K, Paschke E, Jansen E, Jakobs C, et al. Pyridoxine-dependent Epilepsy: Normal Outcome in a Patient with Late Diagnosis after Prolonged Status Epilepticus Causing Cortical Blindness. Neuropediatrics. 2008 Oct 17;39(05):276–9. doi:10.1055/s-0029-1202833
  8. Mills PB, Footitt EJ, Mills KA, Tuschl K, Aylett S, Varadkar S, et al. Genotypic and phenotypic spectrum of pyridoxine-dependent epilepsy (ALDH7A1 deficiency). Brain. 2010 Jul;133(7):2148–59. doi:10.1093/brain/awq143
  9. Ramos RJ, Pras-Raves ML, Gerrits J, van der Ham M, Willemsen M, Prinsen H, et al. Vitamin B6 is essential for serine de novo biosynthesis. J Inherit Metab Dis. 2017 Nov 11;40(6):883–91. doi: 10.1007/s10545-017-0061-3.
  10. de Koning TJ, Duran M, Dorland L, Gooskens R, van Schaftingen E, Jaeken J, et al. Beneficial effects ofL-serine and glycine in the management of seizures in 3-phosphoglycerate dehydrogenase deficiency. Ann Neurol. 1998 Aug;44(2):261–5. doi: 10.1002/ana.410440219.
  11. Jensen KV, Frid M, Stödberg T, Barbaro M, Wedell A, Christensen M, et al. Diagnostic pitfalls in vitamin B6‐dependent epilepsy caused by mutations in the PLPBP gene. JIMD Rep. 2019 Nov 30;50(1):1–8. doi: 10.1002/jmd2.12063
  12. Ware TL, Earl J, Salomons GS, Struys EA, Peters HL, Howell KB, et al. Typical and atypical phenotypes of PNPO deficiency with elevated CSF and plasma pyridoxamine on treatment. Dev Med Child Neurol. 2014 May;56(5):498–502. doi:10.1111/dmcn.12346
  13. Mills PB, Camuzeaux SSM, Footitt EJ, Mills KA, Gissen P, Fisher L, et al. Epilepsy due to PNPO mutations: genotype, environment and treatment affect presentation and outcome. Brain. 2014 May 1;137(5):1350–60. doi: 10.1093/brain/awu051
  14. Regier DS, Greene CL. Phenylalanine Hydroxylase Deficiency. In: Adam MP, Ardinger HH, Pagon RA, et al., editors. GeneReviews® [Internet]. Seattle (WA): University of Washington, Seattle; 1993-2021.
  15. van Karnebeek CDM, Stockler-Ipsiroglu S, Jaggumantri S, Assmann B, Baxter P, Buhas D, et al. Lysine-Restricted Diet as Adjunct Therapy for Pyridoxine-Dependent Epilepsy: The PDE Consortium Consensus Recommendations. In 2014. p. 1–11. doi: 10.1007/8904-2014-296
  16. Stockler S, Plecko B, Gospe SM, Coulter-Mackie M, Connolly M, van Karnebeek C, et al. Pyridoxine dependent epilepsy and antiquitin deficiency. Mol Genet Metab. 2011 Sep;104(1–2):48–60. doi: 10.1016/j.ymgme.2011.05.014.
  17. Kölker S, Christensen E, Leonard J V., Greenberg CR, Boneh A, Burlina AB, et al. Diagnosis and management of glutaric aciduria type I – revised recommendations. J Inherit Metab Dis. 2011 Jun 23;34(3):677–94. doi: 10.1007/s10545-011-9289-5
  18. Mathis D, Abela L, Albersen M, Bürer C, Crowther L, Beese K, et al. The value of plasma vitamin B6 profiles in early onset epileptic encephalopathies. J Inherit Metab Dis. 2016 Sep 24;39(5):733–41. doi: 10.1007/s10545-016-9955-8.
  19. Kozhanova T V., Zhilina SS, Mescheryakova TI, Lukyanova EG, Osipova K V., Ayvazyan SO, et al. Mutation of the ALDH7A1 gene in a patient with pyridoxal phosphate-dependent neonatal epileptic encephalopathy: a clinical case. Epilepsia paroxysmal Cond. 2019 Apr 27;11(1):70–8. (in Russ) DOI: 10.17749/2077-8333.2019.11.1.70-78.