Page: Previous  1, 2, 3  Next

techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3749
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
nice work, I'll watch this, and once you feel it's stable, add it in.

I was expecting a full rewrite of this script from someone but they vaporized so I guess I'll start doing the maintaining again.

First to add is better support for opts, specifically case sensitivity, I have the code for that, but was waiting for the new version, which I guess is now officially not happening.
Back to top
etique
Status: Curious
Joined: 27 Oct 2011
Posts: 7
Location: France
Reply Quote
Ah ah, glad you like !

Well, unfortunately I had to gave up on multithreading...

Just to use forking instead :)

It was really easy actually, here's a version that works on a dual core/threads. We could think of adding a parameter like "$cpu_cores".

:: Code ::
sub convert_file {
my $result; my $cmd;
my ($inFile, $outFile) = @_;

   if ( $OUTPUT_TYPE eq 'ogg' ){
      $result = `$COMMAND_OGG $SILENT_ARG_FO -q $QUALITY -o "$outFile" "$inFile"`;
   }
   elsif ( $OUTPUT_TYPE eq 'mp3' ){
      # Modified: Odd @2011-03-23 01:51:26 - Include ID3 copying
      $cmd = &flac2mp3_cmd( $COMMAND_FLAC, "$inFile", $COMMAND_LAME, $QUALITY, "$outFile" );
      $result = `$cmd`;
   }
   return $result;
}

sub sync_collection_files {
   my $file; @files; $destinationFile; $srcInfo; $srcModTime; $destInfo; $destModTime;
   my $pm = new Parallel::ForkManager(2);
   my $inFile; $outFile; $bFileCreated;
   @files = @_;
   $file = '';
   $outFile = '';
   $inFile = '';
   $destinationFile = '';
   $bFileCreated = '0';
   $srcInfo = '';
   $srcModTime = '';
   $destInfo = '';
   $destModTime = '';

   foreach $file (@files) {
      $file =~ s/\n$//;
      $file =~ s/^\.\///;
      #print "F: $DIR_PREFIX_DEST/$file\n";
      
      # Figure out what the destination file would be...
      $destinationFile = $file;
      if ( $EXTENSION eq $INPUT_TYPE ){
         $destinationFile =~ s/\.$INPUT_TYPE$/\.$OUTPUT_TYPE/;
      }
      #print "D: $destinationFile\n";

      # Now stat the destinationFile, and see if it's date is more recent
      # than that of the original file. If so, we re-encode.
      # We also re-encode if the user supplied --force
      $srcInfo = stat ("$DIR_PREFIX_SOURCE/$file");

      $srcModTime = $srcInfo->mtime;
      $destInfo = stat ("$DIR_PREFIX_DEST/$destinationFile");
      if ( $destInfo ) {
         $destModTime = $destInfo->mtime;
         # print "DEST_MOD: $destModTime :: SRC_MOD: $srcModTime :: FORCE: $force\n";
#       } else {
#          print "NOT EXISTS: $destinationFile \n";
#          print "P1: $file ==> \n\t$destinationFile\n";
      }
      # If the destination file does not exist, or the user specified force,
      # or the srcfile is more recent then the dest file, we encode.
      if ( !$destInfo || $B_FORCE || ( $srcModTime > $destModTime) ) {
         $file =~ s/\`/\'/g;
         $inFile = "$DIR_PREFIX_SOURCE/$file" . "c";
         chop ($inFile);
         $outFile = "$DIR_PREFIX_DEST/$destinationFile" . "g";
         chop ($outFile);
         eval $PRINT_LINE_SMALL;
         if ( $EXTENSION eq $INPUT_TYPE ){
            if ( ! $B_SILENT ) {
               print "ENCODE: $file ==> \n";
               print "        $destinationFile\n";
            }
            else {
               # add line break only when file exists
               print "\nEncoding $file to $OUTPUT_TYPE...";
            }
            $inFile =~ s/\0//g;
            $outFile =~ s/\0//g;

            $pm->start and next; # do the fork

#            if ( $OUTPUT_TYPE eq 'ogg' ){
#               $cmd = "$COMMAND_OGG $SILENT_ARG_FO -q $QUALITY -o \"$outFile\" \"$inFile\"";
#            }
#            elsif ( $OUTPUT_TYPE eq 'mp3' ){
#               # Modified: Odd @2011-03-23 01:51:26 - Include ID3 copying
#               $cmd = &flac2mp3_cmd( $COMMAND_FLAC, "$inFile", $COMMAND_LAME, $QUALITY, "$outFile" );
#               #$result = `$cmd`;
#            }

               &convert_file($inFile, $outFile);

            $pm->finish;
         }
         else {
            if ( ! $B_SILENT ) {
               print "COPY: $file ==> \n";
               print "      $outFile\n";
            }
            else {
               print "\nCopying $file...";
            }
            $inFile =~ s/\0//g;
            $outFile =~ s/\0//g;
            $result = `cp -f "$inFile" "$outFile"`;
         }
         $bFileCreated = 'true';
         $B_DEST_CHANGED = 'true';
      }
   }
   if ( ! $bFileCreated ){
      if ( ! $B_SILENT ) {
         print "No files to process of type: $EXTENSION\n";
      }
      else {
         print "none found.";
      }
   }

   # wait for all the forks to finish before terminating
   # the parent.. otherwise terminating the parent force kills
   # all the forks
   $pm->wait_all_children;
}


And you need to add a line "use Parallel::ForkManager;" at the beginning (and add the proper Fork module... wasn't there by default on Ubuntu 11.10)

Hope you like!
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3749
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
Oh, I was going to note, no modules that are not in default perl install should be used, but if they are, for multithreading, the logic needs to test for that module and give error to let users know what to do.

It's actually fine for multithread to have a non base install module needed since that's a more specialized feature.

In general I'd like to keep acxi as something you download and run as a script, and it will of course tell you if you are asking for a codec operation where the app needed is absent. So the same would go with an advanced feature, it would need to test for that module and give error and instructions on what to do if it's been requested. Not sure how you make perl do that.

I'll take a look at the code tomorrow , thanks for the work.
Back to top
etique
Status: Curious
Joined: 27 Oct 2011
Posts: 7
Location: France
Reply Quote
Mmm that's a good point. Are we able to test the presence of a module in perl and act in consequence?

I think it's still worth having such an option. People who know about multi-core will be interested in this, especially if we describe it in the comments.

Just to recap, we need:
- a parameter to choose for a 1->1 library replication (thus allow a cleaning procedure)
- to test the presence of the Pallel::ForkManager and have a small if allowing for a forking or standard approach
- to add a $cpu_core parameter to set the number of possible forks
Back to top
inytar
Status: New User - Welcome
Joined: 10 Feb 2012
Posts: 1
Reply Quote
I made to patches for the script:

The first one is very simple: It will let you encode ogg files from quality -1 till 10 see [link].

:: Code ::
160c160
<    print "--quality n -q n  For ogg: n can be 1-10, 10 is the biggest file/highest quality.\n";
---
>    print "--quality n -q n  For ogg: n can be -1-10, 10 is the biggest file/highest quality.\n";
352c352
<    if ( $OUTPUT_TYPE eq 'ogg' && $QUALITY !~ m/^([1-9]|10)$/ ) {
---
>    if ( $OUTPUT_TYPE eq 'ogg' && $QUALITY !~ m/^(-[0-1]|(\d|10))$/ ) {
354c354
<       $errorMessage = "$errorMessage\n\t$OUTPUT_TYPE only supports 1-10 quality. You entered: $QUALITY\n";
---
>       $errorMessage = "$errorMessage\n\t$OUTPUT_TYPE only supports -1-10 quality. You entered: $QUALITY\n";


I also wanted to add the possibility to use fractional qualities, but it seems oggenc processes the fractions with a , (comma) or a . (dot) depending on your locale and I don't know how to easily implement that.

The other patch makes it possible to copy playlists.

:: Code ::
75a76
> our @COPY_PLAYLISTS;
124a126,131
> #
> # Playlists types to copy over to the new directories. Has been
> # tested with both .m3u and .cue files with relative paths in the
> # files. Will not work (yet) with absolute paths or relative paths
> # outside of your $DIR_PREFIX_SOURCE.
> @COPY_PLAYLISTS = ( 'cue', 'm3u' );
504c511,521
<          }
---
>          }
>             elsif ( "@COPY_PLAYLISTS" =~ /(^|\s)$EXTENSION(\s|$)/ ) {
>                 &sync_playlist_files ("$inFile", "$outFile" );
>             if ( $LOG_LEVEL > 1 ) {
>                print "COPY: $file ==> \n";
>                print "      $outFile\n";
>             }
>             elsif ( $LOG_LEVEL > 0 ) {
>                print "\nCopying $file...";
>             }
>             }
555a573,590
> sub sync_playlist_files {
>     my ($inPlaylist, $outPlaylist) = @_;
>     my @outData;
>     open (INFILE, "$inPlaylist");
>     while (<INFILE>) {
>         $_ =~ s/\.$INPUT_TYPE/\.$OUTPUT_TYPE/;
>         if ( $OUTPUT_TYPE eq 'mp3' && $EXTENSION eq 'cue' && $_ =~ m/^FILE/) {
>             $_ =~ s/WAVE$/MP3/;
>         }
>         @outData = (@outData,"$_");
>     }
>     close INFILE;
>     open (OUTFILE, ">$outPlaylist");
>     print OUTFILE join("",@outData);
>     close OUTFILE;
>     
> }
>
561c596
<    @extensions = ( @COPY_TYPES, $INPUT_TYPE );
---
>    @extensions = ( @COPY_TYPES, $INPUT_TYPE, @COPY_PLAYLISTS );


I've got it working for both cue and m3u files and they have to use a relative path to point to the music files. I guess it should also work with pls files, but I don't have any to test with.

This is my first time doing anything in perl, and any comments, corrections and improvements are very welcome.
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3749
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
looks ilke good ideas, don't worry about first time with perl, acxi is also my only perl project, it's the ideas and effort and code, m ost important, that matter.
Back to top
Oddlid
Status: New User - Welcome
Joined: 17 Mar 2012
Posts: 2
Location: Sverige
Reply Quote
Hi!
Sorry for the long silence. Very busy times.
I've been playing a bit with multithreading in the new version, but I think I'll rather go with forking instead, as Perl threads are not lightweight, and creates a full copy of the program for each thread. But it tends to go some months between each time I have a look at the script, and then I usually rewrite the whole think, so I can't really give an estimate on when it's done and how it'll be, unless someone sees it as very urgent...
But I haven't forgotten about it, at least :)

BR
Odd
Back to top
chibi
Status: New User - Welcome
Joined: 01 Nov 2012
Posts: 1
Reply Quote
hi,

thanks for the great tool - exactly what i was looking for. how do I force the script to use utf-8? I'm using cygwin to run this in a windows environment, but if there are non-standard characters it seems to create an error. or does it have to be set in cygwin? here is a sample error:

:: Quote ::
Encoding Helrunar/Grátr/01 - Der Fährtensucher.flac to ogg...ERROR: Input file "/cygdrive/m/TEST/FLAC/Helrunar/Grátr/01 - Der Fährtensucher.flac" is not a supported format


"Grátr" contains a character that is apparently not in the charset that is being used or something. if there are no special characters in the filename, then it works fine.

thanks for your help
Back to top
Oddlid
Status: New User - Welcome
Joined: 17 Mar 2012
Posts: 2
Location: Sverige
Reply Quote
Hi,

This error seems to come from the underlying tools, as this is not a messsage from acxi itself, at least not the versions I have.
It might be a problem with the cygwin environment, trying to use utf8, while windows itself uses windows-1232. If possible, you could test by booting some live linux distro and run acxi on the same filesystem, but from linux and not cygwin.
Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 3749
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
cygwin.com/cygwin-ug-net/setup-locale.html

:: Quote ::
language[[_TERRITORY][.charset][@modifier]]

"language" is a lowercase two character string per ISO 639-1, or, if there is no ISO 639-1 code for the language (for instance, "Lower Sorbian"), a three character string per ISO 639-3.

"TERRITORY" is an uppercase two character string per ISO 3166, charset is one of a list of supported character sets. The modifier doesn't matter here (though some are recognized, see below). If you're interested in the exact description, you can find it in the online publication of the POSIX manual pages on the homepage of the Open Group.

Typical locale specifiers are

"de_CH" language = German, territory = Switzerland, default charset
"fr_FR.UTF-8" language = french, territory = France, charset = UTF-8
"ko_KR.eucKR" language = korean, territory = South Korea, charset = eucKR
"syr_SY" language = Syriac, territory = Syria, default charset

If the locale specifier does not follow the above form, Cygwin checks if the locale is one of the locale aliases defined in the file /usr/share/locale/locale.alias. If so, and if the replacement localename is supported by the underlying Windows, the locale is accepted, too. So, given the default content of the /usr/share/locale/locale.alias file, the below examples would be valid locale specifiers as well.

"catalan" defined as "ca_ES.ISO-8859-1" in locale.alias
"japanese" defined as "ja_JP.eucJP" in locale.alias
"turkish" defined as "tr_TR.ISO-8859-9" in locale.alias

The file /usr/share/locale/locale.alias is provided by the gettext package under Cygwin.

At application startup, the application's locale is set to the default "C" or "POSIX" locale. Under Cygwin 1.7.2 and later, this locale defaults to the ASCII character set on the application level. If you want to stick to the "C" locale and only change to another charset, you can define this by setting one of the locale environment variables to "C.charset". For instance


cygwin.com/faq-nochunks.html#faq.using.unicode
:: Quote ::
4.17.

Why don't international (Unicode) characters work?

Internationalization is a complex issue. The short answer is that Cygwin relies on the setting of the setting of LANG/LC_xxx environment variables. The long answer can be found in the User's Guide in the section Internationalization

Cygwin uses UTF-8 by default. To use a different character set, you need to set the LC_ALL, LC_CTYPE or LANG environment variables.


oddlid has it right, you created the files in windows, which is using the non standard 1252 character set. It's best in general to switch everything to use utf8, that way it all works well together. But you can't easily convert file names etc, sadly.

However, you can use rename, which may or may not be in cygwin, to easily do that, you simply run it in each directory where non utf8 characters exist, and replace them with the utf8 version. Well, not totally simple, but it is doable with some patience. But that's only sensible if you already have converted teh base system to use utf8. Does windows have that option? I don't know.

I believe though I'm not certain that you can also set these in Perl, inside of acxi, that's a common method, that way you don't change your base system, but acxi/cygwin will use whatever you tell it to use.
Back to top
Display posts from previous:   
Page: Previous  1, 2, 3  Next
All times are GMT - 8 Hours