This situation is an example of Bombay phenomenon. The antigen for blood group (A or B) actually attaches to another molecule encoded by gene H. Most humans have at least one H allele. However, in the rare cases, there is a hh genotype, which makes the expression of blood group allele impossible. A person will "look like" O group even in spite of the fact that he/she has allele for A or B antigen.
In case of this family, the children had to receive B allele from the mother. However, they do not have it expressed. It means that they have hh genotype. They had to receive one h allele from father and one h allele from mother. As the mother has a blood group B, she also has a copy of the dominant allele H. Thus, her genotype is BBHh. In case of the father, it is not clear. He could have either OOhh or OOHh genotype. In the first case, the likelihood for the outcome observed in children is higher. The children have a genotype BOhh, which results in blood group O.