• Antonino Daplas's avatar
    [PATCH] fbcon: "Do not touch hardware if vc_mode != KD_TEXT: fix · fce79446
    Antonino Daplas authored
    Sigh, this patch uncovered a can of worms.  I tested different combinations,
    those with/without xxxfb_blank implementation, framebuffers in directcolor
    or truecolor, etc. I find that there is a problem unblanking if the hardware
    has no xxxfb_blank() implementation, and also that the generic fb_blank() in
    fbmem.c is problematic with drivers in directcolor or pseudocolor mode when
    called by an fb application such as X.
    
    Display blanking is implemented in three ways:
    
        1. using the drivers blanking implementation - info->fbops->fb_blank()
        2. clearing the screen with the console erase character - fbcon_blank()
        3. setting the color map to all black - fb_blank()
    
    The third method is problematic for these reasons:
    
        - Setting the colormap to all black will not work in truecolor mode
        - In directcolor or pseudocolor, it will overwrite the fb application's
          color map, producing wrong colors.
    
    So, remove the generic implementation in fb_blank() and just return -EINVAL
    if there is no hardware implementation.  This will be only used by apps doing
    an FBIO_BLANK ioctl, and is a more robust approach.
    
    Other changes:
    
    - Consolidated all tests and created an inlined helper function to check if
      the framebuffer console is inactive or not.
    - fix unblanking if driver has no xxxfb_blank() hook.
    
    I'm probably missing a few more things, but this patch should be generally
    correct. Please apply after the entire patch series to avoid rejects.
    Signed-off-by: default avatarAntonino Daplas <adaplas@pol.net>
    Signed-off-by: default avatarAndrew Morton <akpm@osdl.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@osdl.org>
    fce79446
fbmem.c 30.5 KB