[IDEA] Genius

[kit1cat]

Would it be possible to add Genius.com to mode 7 search engine options? Search for album/artist?

[Tim Curtis]

What is the syntax of the search query?

[kit1cat]

Sorry Tim, not sure what you mean?

[Stephanowicz]

What is the syntax of the search query?

Hi,

I tried several api-queries - it's not this easy to find a good & reliable one.
Finally I found genius-lyrics-api by farshed. It's using node.

I think it has everything of the possible queries available. 
It's a bit lazy - but this could also be because of the overloaded Genius server...

I created a php script and a node script for querying the actual song playing
In order to use it, You will need to create a free account on Genius. Then You need to create a new API-Client in order to get the API-Key.

PHP Code:

<?php

$ARTIST_=shell_exec('mpc --format %artist% | head -n 1');
$ARTIST_=str_replace("\n","",$ARTIST_);
$TITLE_=shell_exec('mpc --format %title% | head -n 1');
$TITLE_=str_replace("\n","",$TITLE_);

$ret = shell_exec("node /home/pi/node/geniuslyrics.js '".$ARTIST_."' '".$TITLE_."'");
print_r($ret); 


Code:

const { getLyrics, getSong  } = require('genius-lyrics-api');

if (process.argv.length==4) {

var songtitle = process.argv[3]
var songartist = process.argv[2]

const options = {
apiKey: '>>>>YOUR API KEY<<<<',
title: songtitle,
artist: songartist,
optimizeQuery: true
};

getLyrics(options).then((lyrics) => console.log(lyrics));

};

//getSong(options).then((song) =>
// console.log(`
// ${song.id}
// ${song.url}
// ${song.albumArt}
// ${song.lyrics}`)
//);

here's a post describing to query the API with python: Getting Song Lyrics from Genius’s API + Scraping | Big-Ish Data


As I mentioned elswhere in my X-Mas wishlist, it would be really great to have lyrics included!

Cheers, Stephan

[kit1cat]

Thanks Stephan, Tim’s question was beyond me.

[Stephanowicz]

I tested then link from above with the python script.
Needed some fixes, but finally got it working - and it is much faster than the node solution I posted above.
But maybe the aprroach is not the best as it searches first for a song title... well, if this title is quite common I think You know what will happen... :

EDIT: I found out that You can add the artist to the songtitle query - so the query is nicely narrowed down
EDIT: also added a query for the song that is currently playing in mpd

So - for those who are interested:

-In the line at the beginning: 
headers = {'Authorization': 'Bearer <<<YOUR Client Access Token>>>'}
You will have to enter Your Client Access Token

I left some prints so You can see what's happening

Code:

#!/usr/bin/python3
# -*- encoding: utf-8 -*-

import requests
from bs4 import BeautifulSoup
from os import popen
from time import sleep
base_url = "https://api.genius.com"
headers = {'Authorization': 'Bearer <<<YOUR Client Access Token>>>'}
artist_name = popen('mpc --format %artist% | head -n 1').read().replace("\n","")
song_title = popen('mpc --format %title% | head -n 1').read().replace("\n","")

def lyrics_from_song_api_path(song_api_path):
  song_url = base_url + song_api_path
  response = requests.get(song_url, headers=headers)
  json = response.json()
  path = json["response"]["song"]["path"]
  #gotta go regular html scraping... come on Genius
  page_url = "https://genius.com" + path
  print(page_url)
  print()
  print()
  res= None
  i=0
  while res == None:
    if i > 5: break
    page = requests.get(page_url)
    html = BeautifulSoup(page.text, "html.parser")
  #remove script tags that they put in the middle of the lyrics
#  [h.extract() for h in html('script')]
  #at least Genius is nice and has a tag called 'lyrics'!
    res = html.find('div', class_='lyrics')
    i+=1
    sleep(1)
  if res:  
    lyrics = res.get_text() #updated css where the lyrics are based in HTML
    return lyrics
  else:
    return "Found lyrics page but unable to query the lyrics"
if __name__ == "__main__":
  search_url = base_url + "/search"
  data = {'q': song_title + ' ' + artist_name}
  response = requests.get(search_url, params=data, headers=headers)
  print(response.url)
  json = response.json()
  song_info = None
  for hit in json["response"]["hits"]:
    if hit["result"]["primary_artist"]["name"] == artist_name:
      song_info = hit
      break
    else:
      print(hit)
  if song_info:
    song_api_path = song_info["result"]["api_path"]
    print(song_api_path)
    print()
    print(artist_name + " - " + song_title)
    print (lyrics_from_song_api_path(song_api_path))

Cheers, Stephan

EDIT: while the query is well narrowed down, there was still a problem with requesting the final page containing the lyrics - dunno what's happening, maybe the page is not completely loaded - but, I created a loop which tries to receive the content in several attempts... seems to work atm

[Stephanowicz]

Ok, so here is a pure php-script!

It's based on the python script mentioned above. And I found a working function by Allen Fair which creates the api-call.

For the parsing of the lyrics page I used DOMXPath as replacement for the python function BeautifulSoup.

As DOMXPath (php xml) is not installed You have to install it manually with:

Code:

sudo apt-get install php7.3-xml 

then You may copy the code in a new file in /var/www/command/ - let's call it geniuslyrics.php
You need to put Your Genius client access token in the first line ($ClientAccessToken = '....')
When playing a song You then can open the webpage with http://moode.local/command/geniuslyrics.php 

PHP Code:

<?php
$ClientAccessToken = '<<<PUT YOUR CLIENT ACCESS TOKEN HERE>>';
$ARTIST_=shell_exec('mpc --format %artist% | head -n 1');
$ARTIST_=str_replace("\n","",$ARTIST_);
$ARTIST=str_replace(" ","+",$ARTIST_);

$TITLE_=shell_exec('mpc --format %title% | head -n 1');
$TITLE_=str_replace("\n","",$TITLE_);
$TITLE=str_replace(" ","+",$TITLE_);

function http_request($url, $opt=array()) {
/**
 * Makes a remote HTTP request and returns the response.
 *
 * @author    Allen Fair <allen.fair@gmail.com>
 * @copyright 2018 Allen Fair
 * @license   https://opensource.org/licenses/MIT  MIT
 * @code      https://gist.github.com/afair/a7c7adc52b7b49bf362935e665a87633
 *
 * @param string  $url     The full URL endpoint: "https://example.com/endpoint"
 * @param array   $opt     An Associative Array of extended request options
 *
 *        string  'method'       one of: GET, POST, PUT, PATCH, DELETE, HEAD
 *        string  'url'          URL override
 *        string  'endpoint'     Optional endpoint appended to URL
 *        string  'user'         user name to send for HTTP Basic Authorization
 *        string  'token'        creates an "Authorization: Bearer" header
 *        string  'accept'       creates an Accept: header for the mime/type
 *        string  'password'     password to send for HTTP Basic Authorization
 *        array   'headers'      array of "Header-Name: value", ...
 *        array   'query'        name=>value pairs for the Query String
 *        array   'post'         name=>value pairs for the POST Data body
 *        array   'files'        name=>['path'=>"/path/file", 'filename'=>"name.ext",
 *                                     'type'=>"mime1/mime2"] pairs of files to send
 *        array   'cookies'      name=value pairs for cookies to send
 *        array   'http_options' name=value pairs for "HTTP context options"
 *        callable 'notifications' a stream_notification_callback for events on the stream.
 *
 * @return array  [$http_status_code, $headers_hash, $body]
 *
 * Note: sending very large files/requests must fit into memory. Chunking is not supported.
 *
 * @example
 * list($status, $headers, $content) = http_request('https://www.example.com/endpoint', array(
 *   'method'       => 'POST',
 *   'headers'      => array('From'=>'pat@mycompany.com'),
 *   'accept'       => 'application/json',
 *   'user'         => $apikey, 'password'=>'', 'token'=>$token, // Authentication
 *   'query'        => array('q' => 'PHP HTTP request'),
 *   'post'         => array('email' => 'pat@mycompany.com'),
 *   'files'        => array('path'=> './avatar.png', 'filename'=>'avatar.png', 'type'=>'image/png'),
 *   'cookies'      => array('session' => $jwt),
 *   'http_options' => array('timeout'=> 5.0, 'user_agent'=>"php_request/1.0"),
 *   ));
 */
  $http_options = array_key_exists('http_options', $opt) ? $opt['http_options'] : array();
  $headers      = array_key_exists('headers', $opt) ? $opt['headers'] : array();
  $url          = array_key_exists('url', $opt) ? $opt['url'] : $url;
  $endpoint     = array_key_exists('endpoint', $opt) ? $opt['endpoint'] : null;
  $method       = array_key_exists('method', $opt) ? strtoupper($opt['method']) : 'GET';
  $query        = array_key_exists('query', $opt) ? $opt['query'] : null;
  $post         = array_key_exists('post', $opt) ? $opt['post'] : null;
  $files        = array_key_exists('files', $opt) ? $opt['files'] : null;
  $notifications= array_key_exists('notifications', $opt) ? $opt['notifications'] : null;

  // Headers
  if (array_key_exists('accept', $opt)) {
    $headers[] = "Accept: " . $opt['accept'];
  }
  if (array_key_exists('user', $opt)) {
    $headers[] = "Authorization: Basic ".base64_encode($opt['user'].':'.$opt['password']);
  }
  elseif (array_key_exists('token', $opt)) {
    $headers[] = "Authorization: Bearer ".$opt['token'];
  }
  if (array_key_exists('cookies', $opt)) {
    foreach ($opt['cookies'] as $n=>$v) $headers[] = "Cookie: $n=$v";
  }

  // URL, appended Endpoint, and Query Parameters
  if ($endpoint) $url .= $endpoint;
  if (!empty($query)) {
    $url .= (strpos("?",$url)===false ? "?" : "&") . http_build_query($query);
  }
  // File + Post Parameters in multipart format
  if (!empty($files)) {
    $boundary = '--------------------------'.microtime(true);
    $content = array();
    foreach ($files as $field=>$f) {
      $content[] ="--$boundary";
      $content[] = "Content-Disposition: form-data; name=\"$field\"; filename=\"".basename($filename)."\"";
      $content[] = "Content-Type: {$f['type']}";
      $content[] = "Content-Length: " . filesize($f['path']);
      $content[] = "";
      $content[] = file_get_contents($f['path']);
      $content[] = "";
    }
    foreach ($post as $field=>$v) {
      $content[] ="--$boundary";
      $content[] = "Content-Disposition: form-data; name=\"$field\"";
      $content[] = "";
      $content[] = $v;
      $content[] = "";
    }
    $content[] = "--$boundary--";
    $headers[] = "Content-Type: multipart/form-data; boundary=$boundary";
    $http_options['content'] = implode("\r\n", $content);
    unset($content);
  }
  // Post Parameters in urlencoded format
  elseif (!empty($post)) {
    $headers[] = "Content-Type: application/x-www-form-urlencoded";
    $http_options['content'] = http_build_query($post);
  }

  // Make Request
  if (!$http_options['user_agent']) $http_options['user_agent'] = "http_request.php/1.0";
  $http_options['method'] = $method;
  if ($http_options['content']) $headers[] = "Content-Length: ".strlen($http_options['content']);
  if (!empty($headers)) {
    $http_options['header'] = implode("\r\n", $headers);
  }
  $context = stream_context_create(array('http'=>$http_options));
  if ($notifications) {
    stream_context_set_params($context, array("notification" => $notifications));
  }
  if ($method == 'HEAD') {
    $headlist = @get_headers($url, 0, $context);
    $response = null;
  } else {
    $response = @file_get_contents($url, false, $context);
    $headlist = $http_response_header;
  }

  // Format Response
  $status = preg_match("/(\d\d\d) (.+)/", $headlist[0], $m) ? $m[1] : 500;
  $headers = array("Status" => $headlist[0]);
  foreach ($headlist as $h) {
    if (preg_match("/^([\w\-]+):\s*(.+)/", $h, $m)) $headers[$m[1]] = $m[2];
  }
  return array($status, $headers, $response);
}

?>
<!DOCTYPE html>
<html lang="en">
<head>
    <title>Genius lyrics query</title>
</head>
<body>
<?php
if($TITLE!="" && $ARTIST!=""){
  $result = (http_request('https://api.genius.com/search', array(
   'method'       => 'GET',
   'accept'       => 'application/json',
   'token'         => $ClientAccessToken, // Authentication
   'query'        => array('q' => $TITLE.' '.$ARTIST),
   'http_options' => array('timeout'=> 5.0, 'user_agent'=>"php_request/1.0"),
   )));

  $json = json_decode($result[2], TRUE);
  // print_r($json);
  $val = $json["response"]["hits"];
  // print_r($val);
  $song_api_path="";
  $song_url="";
  foreach($val as $hit){
    if(stripos ($hit["result"]["primary_artist"]["name"],$ARTIST_)!==False){
        //print('found' . chr(10));
        $artistFound=$hit["result"]["primary_artist"]["name"];
        $songFound=$hit["result"]["title"];
        $song_api_path = $hit["result"]["api_path"];
        $song_url=$hit["result"]["url"];
        break;
    };
  };
  if($song_url != ""){
    $songlyrics="";
    $found=false;
    $i=0;
    while(!$found){
      $file = file_get_contents($song_url);
//      file_put_contents ('tmp',$file);
      $file =str_replace("<br>","",$file);
      $dom = new DOMDocument;
      $dom->loadHTML($file);
      $xpath = new DOMXPath($dom);
      foreach( $xpath->query('//div[@class="lyrics"]') as $e ) {
         $songlyrics=$e->nodeValue;
         $found=true;
      };
      $i++;
      if($i>10){break;};
    };
    if($songlyrics!=""){
      $songlyrics=nl2br($songlyrics);
      $songlyrics=str_replace("]","]<br />",$songlyrics);
      $songlyrics=str_replace("[","<br />[",$songlyrics);
      echo $artistFound.' - '.$songFound;
      echo $songlyrics;
    }
  }
  else{echo "no lyrics found..." . chr(10);};

//
}
else{echo "please start playback..." . chr(10);}

?>
</body>
</html> 


Hope it is useful and maybe even gets integrated in moode ui someday...

Have a nice Christmas!

Cheers, Stephan

[Tim Curtis]

I would suggest posting this as an issue in the Git repo. That way it will be easy for me to check it out after the holidays.

[Stephanowicz]

Ok, so we tried... and 'failed' - sorry, but the TOS of Genius doesn't allow us to display 'their' lyrics seperatly in other applications.
(the music industry is trying to make money of everything - I wonder how much of this really gets in to the artists.)
(one Idea was to ask Genius for special permission for moOde... )

The work was ready to get into the repo - You may take a look here: https://github.com/moode-player/moode/pull/299

Cheers, Stephan

[kit1cat]

I use the Genius app on my iPad, could we not click on the song in moode and have the lyrics displayed in the Genius app? To be honest it only takes a few clicks to have the album displayed in Genius and playing in moode on my iPad.