Objective To investigate the relationship between microstructure characteristics and its mechanical behavior. Methods The compression experiments on longitudinal and transverse specimens of the cortical bone from bovine femoral bone were conducted. The fracture routes of longitudinal and transverse specimens and microstructural features of fractured surfaces were observed. The effects of osteon orientation on elastic modulus and ultimate fracture strength of the bone were analyzed based on the theory of composite micromechanics. Results The compressive elastic modulus and ultimate fracture strength of longitudinal specimens were significantly greater than those of transverse specimens. The fracture routes of longitudinal specimens were along their loading direction (0°) and relatively flat, while the fracture routes of transverse specimens were relatively zigzagging, being approximately 45° with their loading direction. There were many striped structures on fractured surfaces of longitudinal specimens and the fractured surfaces were relatively smooth, while there were many annular concave and convex structures on fractured surfaces of transverse specimens and the fractured surfaces were relatively rough. Conclusions The bovine femoral bone had anisotropic mechanical properties, and its anisotropic mechanical properties were closely related to the orientation of osteons.